-- Logs begin at Thu 2022-08-18 15:54:06 UTC, end at Thu 2022-08-18 17:42:13 UTC. -- Aug 18 16:52:49.297640 dv-cinder-853679-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 18 16:52:49.305545 dv-cinder-853679-1-iscsi devstack@keystone.service[81636]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 18 16:52:49.310950 dv-cinder-853679-1-iscsi devstack@keystone.service[81636]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Aug 18 16:52:49 2022] *** Aug 18 16:52:49.310950 dv-cinder-853679-1-iscsi devstack@keystone.service[81636]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 18 16:52:49.310950 dv-cinder-853679-1-iscsi devstack@keystone.service[81636]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 18 16:52:49.310950 dv-cinder-853679-1-iscsi devstack@keystone.service[81636]: nodename: dv-cinder-853679-1-iscsi Aug 18 16:52:49.310950 dv-cinder-853679-1-iscsi devstack@keystone.service[81636]: machine: x86_64 Aug 18 16:52:49.310950 dv-cinder-853679-1-iscsi devstack@keystone.service[81636]: clock source: unix Aug 18 16:52:49.310950 dv-cinder-853679-1-iscsi devstack@keystone.service[81636]: pcre jit disabled Aug 18 16:52:49.310950 dv-cinder-853679-1-iscsi devstack@keystone.service[81636]: detected number of CPU cores: 4 Aug 18 16:52:49.310950 dv-cinder-853679-1-iscsi devstack@keystone.service[81636]: current working directory: / Aug 18 16:52:49.310950 dv-cinder-853679-1-iscsi devstack@keystone.service[81636]: detected binary path: /usr/bin/uwsgi-core Aug 18 16:52:49.310950 dv-cinder-853679-1-iscsi devstack@keystone.service[81636]: your processes number limit is 47680 Aug 18 16:52:49.310950 dv-cinder-853679-1-iscsi devstack@keystone.service[81636]: your memory page size is 4096 bytes Aug 18 16:52:49.310950 dv-cinder-853679-1-iscsi devstack@keystone.service[81636]: detected max file descriptor number: 2048 Aug 18 16:52:49.310950 dv-cinder-853679-1-iscsi devstack@keystone.service[81636]: lock engine: pthread robust mutexes Aug 18 16:52:49.310950 dv-cinder-853679-1-iscsi devstack@keystone.service[81636]: thunder lock: enabled Aug 18 16:52:49.310950 dv-cinder-853679-1-iscsi devstack@keystone.service[81636]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 18 16:52:49.310950 dv-cinder-853679-1-iscsi devstack@keystone.service[81636]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 18 16:52:49.349182 dv-cinder-853679-1-iscsi devstack@keystone.service[81636]: Python main interpreter initialized at 0x55cc9635a9c0 Aug 18 16:52:49.349182 dv-cinder-853679-1-iscsi devstack@keystone.service[81636]: python threads support enabled Aug 18 16:52:49.349182 dv-cinder-853679-1-iscsi devstack@keystone.service[81636]: your server socket listen backlog is limited to 100 connections Aug 18 16:52:49.349182 dv-cinder-853679-1-iscsi devstack@keystone.service[81636]: your mercy for graceful operations on workers is 90 seconds Aug 18 16:52:49.349538 dv-cinder-853679-1-iscsi devstack@keystone.service[81636]: mapped 403077 bytes (393 KB) for 2 cores Aug 18 16:52:49.349614 dv-cinder-853679-1-iscsi devstack@keystone.service[81636]: *** Operational MODE: preforking *** Aug 18 16:52:49.349890 dv-cinder-853679-1-iscsi devstack@keystone.service[81636]: *** uWSGI is running in multiple interpreter mode *** Aug 18 16:52:49.349890 dv-cinder-853679-1-iscsi devstack@keystone.service[81636]: spawned uWSGI master process (pid: 81636) Aug 18 16:52:49.350093 dv-cinder-853679-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 18 16:52:49.355112 dv-cinder-853679-1-iscsi devstack@keystone.service[81636]: spawned uWSGI worker 1 (pid: 81649, cores: 1) Aug 18 16:52:49.355112 dv-cinder-853679-1-iscsi devstack@keystone.service[81636]: spawned uWSGI worker 2 (pid: 81650, cores: 1) Aug 18 16:52:49.355112 dv-cinder-853679-1-iscsi devstack@keystone.service[81636]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 18 16:52:53.689979 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 18 16:52:53.689979 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: from cryptography.utils import int_from_bytes Aug 18 16:52:53.690772 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 18 16:52:53.690772 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: from cryptography.utils import int_from_bytes Aug 18 16:52:53.699768 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 18 16:52:53.699768 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: from cryptography.utils import int_from_bytes Aug 18 16:52:53.700233 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 18 16:52:53.700233 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: from cryptography.utils import int_from_bytes Aug 18 16:52:53.742120 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 18 16:52:53.742294 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 18 16:52:53.742489 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] command line args: [] {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 18 16:52:53.742755 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 18 16:52:53.742953 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] ================================================================================ {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 18 16:52:53.743208 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] admin_token = **** {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.743451 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] config_dir = [] {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.743642 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.743823 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] config_source = [] {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.744002 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] debug = True {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.744198 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.744556 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.744736 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.744986 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.745227 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.745455 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] list_limit = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.745635 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] log_config_append = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.745812 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.745988 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] log_dir = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.746165 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] log_file = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.746394 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.746654 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.746880 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.747157 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.747333 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.747501 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.747733 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.747975 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.748206 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.748434 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.748664 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.748891 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.749133 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.749374 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.749643 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.750092 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 18 16:52:53.750179 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.82/identity {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.750290 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 18 16:52:53.750372 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] publish_errors = False {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.750528 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] command line args: [] {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 18 16:52:53.750660 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.750983 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.751052 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 18 16:52:53.751220 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.751349 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ================================================================================ {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 18 16:52:53.751481 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.751651 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] admin_token = **** {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.751785 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.751930 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] config_dir = [] {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.752063 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] standard_threads = False {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.752196 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.752360 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.752485 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] config_source = [] {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.752485 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] debug = True {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.752740 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.752866 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.753026 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.753026 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] use_journal = False {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.753279 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.753411 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] use_json = False {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.753618 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.753746 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] use_stderr = False {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.753849 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.753946 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] use_syslog = False {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.754534 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.754648 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.754743 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] list_limit = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.754841 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.754954 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] log_config_append = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.759333 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.759479 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.759479 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] log_dir = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.759622 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.759692 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] log_file = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.759768 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.759838 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.759838 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.759971 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.760039 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.760107 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.760180 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.760180 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.760378 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.760449 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.760525 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.760594 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.760673 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.760743 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.760812 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.760880 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.760975 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.761049 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.761128 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.761198 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.761272 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.761348 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.761417 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.761486 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.761567 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.761639 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.761710 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.761784 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.761854 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.761854 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.761995 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.82/identity {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.762065 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.762132 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] publish_errors = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.762132 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.762344 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.762415 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.762488 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.762557 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.762624 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.762710 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.762782 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.762849 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.762991 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.763073 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] standard_threads = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.763154 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.763234 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.763306 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.763382 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.763451 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.763525 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.763525 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] use_journal = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.763657 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.763732 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] use_json = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.763807 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.763877 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] use_stderr = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.763877 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] use_syslog = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.763999 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.764075 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 16:52:53.764151 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.764151 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.764274 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.764347 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.764423 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.764500 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.764570 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.764644 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.764713 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.764786 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.764855 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.764933 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.765008 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.765078 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.765145 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.765221 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.765290 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.765366 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.765436 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.765503 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.765570 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.765643 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.765718 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.765876 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.765954 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.766087 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.766164 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.766234 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.766303 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.766380 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.766450 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.766519 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.766587 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.766655 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.766722 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.766799 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.766869 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.767406 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.767525 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.767633 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.767741 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.767840 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.767840 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.768026 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.768129 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.768242 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.768350 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.768456 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.768554 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.768658 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.768762 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.768860 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.768965 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.769069 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.769167 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.769275 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.769407 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.769514 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.769621 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.769789 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.770402 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.770514 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.770613 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.770730 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.770836 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.770969 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.771080 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.771186 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.771289 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.771388 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.771492 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.771598 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.771714 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.771837 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.771944 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.772057 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.772057 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.772244 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.772356 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.772356 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.772550 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.772661 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.772765 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.772863 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.772975 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.773136 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.773256 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.773372 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.773477 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.773592 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.773716 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.773826 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.773941 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.774056 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.774179 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.774296 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.774423 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.774530 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.774645 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.774760 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.774876 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.775031 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.775151 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.775266 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.775381 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.775502 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.775619 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.775746 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.775864 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.775981 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.776097 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.776211 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.776326 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.776449 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.776567 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.776682 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.776798 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.776914 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.777500 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.777689 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.777778 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.777839 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.777911 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.777970 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.778109 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.778188 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.778259 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.778337 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.778409 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.778490 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.778571 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.778659 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.778743 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.778816 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.778894 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.779045 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.779124 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.779124 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.779259 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.779332 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.779400 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.779522 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.779640 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.779755 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.779879 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.779997 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.780103 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.780203 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] auth.external = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.780302 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.780406 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.780513 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.780513 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.780695 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.780793 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.780892 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.780989 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.781097 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] auth.password = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.781097 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] auth.token = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.781282 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.781379 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.781577 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.781687 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.781799 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.781799 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.781982 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.781982 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.782170 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.782170 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.782367 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.782367 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.782592 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.782708 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.782824 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.782954 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] credential.caching = True {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.783085 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.783239 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.783359 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.783480 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.783597 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.783721 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.783844 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.783961 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.784075 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.784217 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.784344 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.784344 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.784561 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.784676 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.784800 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.784925 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.785041 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.785156 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.785271 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.785386 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.785510 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.785627 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.785743 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.785858 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.786009 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.786208 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.786288 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.786360 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.786433 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.786503 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.786574 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.786653 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.786653 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.786790 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] federation.caching = True {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.786861 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] auth.external = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.786861 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.787049 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.787132 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.787214 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.787287 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.787358 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.787428 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] auth.password = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.787498 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.787579 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] auth.token = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.787652 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.787723 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.787794 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.787864 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.787935 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.788013 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.788086 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.788157 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.788228 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.788298 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.788368 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.788446 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.788519 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.788623 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.788707 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] identity.caching = True {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.788860 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] credential.caching = True {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.788942 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.789115 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.789199 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.789366 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.789448 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.789600 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.789679 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.789840 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.789920 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.790069 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.790147 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.790306 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.790390 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.790551 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.790631 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.790802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.790882 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.791078 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.791168 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.791326 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.791408 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.791586 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.791664 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.791836 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.791915 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.792083 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.792163 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.792327 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.792406 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.792582 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.792667 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.792817 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] federation.caching = True {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.792900 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.793054 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.793133 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.793301 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.793389 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.793533 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.793613 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.793765 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.793844 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.793997 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.794075 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.794228 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.794309 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.794462 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.794543 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.794700 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.794791 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.794957 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.795085 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.795206 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.795322 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.799411 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.799551 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.799630 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.799702 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] identity.caching = True {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.799774 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.799844 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.799916 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.799997 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.799997 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.800128 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.800198 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.800268 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.800338 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.800419 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.800492 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.800561 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.800631 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.800701 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.800770 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.800848 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.801442 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.801567 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.801643 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.801766 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.801842 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.801975 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.802168 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.802393 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.802477 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.802600 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.802716 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.802801 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.803012 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.803099 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.803172 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.803244 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.803358 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.803455 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.803455 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.803590 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.803661 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.803721 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.803844 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.803971 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.804065 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.804139 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.804211 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] ldap.user = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.804281 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.804281 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.804417 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.804490 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.804490 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.804618 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.804688 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.804759 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.804837 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.804837 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.804965 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.805035 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.805035 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.805172 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.805250 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.805250 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.805379 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.805449 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.805449 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.805575 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.805652 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.805725 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.805801 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.805892 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.805970 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.806030 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.806212 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.806297 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.806297 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.806426 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.806497 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.806568 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.806646 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.806726 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.806799 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.806870 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.806969 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.807093 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.807183 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.807259 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.807330 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.807422 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.807497 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.807568 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.807647 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.807721 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.807792 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.807862 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.807933 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.808011 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.808084 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.808155 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.808226 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.808296 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.user = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.808370 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.808452 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.808531 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.808604 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.808675 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.808802 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.808890 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.809034 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.809117 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.809283 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.809364 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.809519 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.809598 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.809753 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.809833 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.809986 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.810066 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.810223 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.810302 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.810452 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.810532 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.810686 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.810770 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.810930 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.811047 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] resource.caching = True {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.811183 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.811267 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.811426 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.811504 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.811662 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.811742 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.811907 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.811987 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.812153 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.812231 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.812381 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.812460 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.812627 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.812709 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.812862 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.812941 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.813098 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.813166 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.813333 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.813446 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] role.caching = True {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.813568 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.813639 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] role.driver = sql {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.813794 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.813861 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.814041 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.814112 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.814277 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.814345 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.814524 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.814592 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.814754 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.814823 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.815023 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.815097 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.815263 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.815333 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.815500 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.815572 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.815736 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.815806 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.815966 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.816034 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.816213 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.816283 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.816444 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.816538 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.817483 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.817578 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.817767 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] resource.caching = True {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.817883 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.818021 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.818273 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.818341 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.818512 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.818589 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.818765 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.818834 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.819038 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.819112 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.819278 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.819361 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.819527 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.819608 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.819761 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.819840 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.820001 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.820083 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.820239 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] role.caching = True {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.820319 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.820480 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.820558 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] role.driver = sql {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.820710 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.820789 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.820937 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.821016 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.821167 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.821247 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.821417 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.821498 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.821662 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.821741 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.821906 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.821984 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.822144 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.822226 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.822379 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.822459 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.822608 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.822688 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.822843 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] token.caching = True {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.822947 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.823104 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.823186 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.823350 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.823429 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.823583 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.823665 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.823829 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.823908 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.824073 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.824245 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.824334 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.824482 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.824562 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.824735 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.824815 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.825000 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.825087 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.825235 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.825316 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.825471 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.825551 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.825704 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.825793 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.825947 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.826035 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.826192 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.826274 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.826420 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.826509 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.826661 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.826740 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.826892 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.827038 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.827146 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.827229 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] database.connection = **** {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.827396 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.827486 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.827651 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.827732 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.827898 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.827977 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.828135 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.828216 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.828369 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.828449 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.828600 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.828680 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.828829 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.828909 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] token.caching = True {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.829063 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.829143 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.829297 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.829381 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.829523 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.829605 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.829764 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.829844 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.829996 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.830076 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.830232 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.830312 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.830468 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.830597 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.830702 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.830782 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.830943 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.831030 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.831185 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.831265 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.831420 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.831558 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.831647 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81650) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 18 16:52:53.831723 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.831896 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.832173 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.832410 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.832659 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.833066 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.833334 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] database.connection = **** {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.833587 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.833837 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.834077 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.834321 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.834562 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.834801 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.835073 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.835312 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.835556 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.835794 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.836038 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.836274 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.836521 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.836756 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.836999 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.837231 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.837471 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.837706 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 16:52:53.837877 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 18 16:52:53.866870 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81650) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 16:52:53.867125 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81650) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 16:52:53.867354 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81650) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 16:52:53.867893 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81649) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 16:52:53.868131 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81649) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 16:52:53.868358 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81649) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 16:52:53.876454 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81650) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 16:52:53.876905 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81649) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 16:52:53.905103 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81650) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 16:52:53.905328 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81650) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 16:52:53.905429 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81649) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 16:52:53.905603 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81649) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 16:52:53.960516 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81649) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 16:52:53.964041 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81650) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 16:52:54.010311 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81649) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 16:52:54.010543 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81649) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 16:52:54.012031 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81650) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 16:52:54.012266 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81650) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 16:52:54.179968 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81650) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 16:52:54.180178 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81650) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 16:52:54.180399 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81650) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 16:52:54.180619 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81650) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 16:52:54.180834 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81650) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 16:52:54.181045 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81650) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 16:52:54.187300 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81649) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 16:52:54.187535 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81649) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 16:52:54.187756 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81649) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 16:52:54.187974 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81649) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 16:52:54.188190 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81649) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 16:52:54.188403 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81649) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 16:52:54.204521 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81650) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 16:52:54.204752 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81650) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 16:52:54.204975 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81650) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 16:52:54.205190 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81650) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 16:52:54.205403 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81650) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 16:52:54.205613 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81650) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 16:52:54.205824 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81650) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 16:52:54.206197 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81649) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 16:52:54.206433 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81649) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 16:52:54.206656 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81649) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 16:52:54.206874 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81649) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 16:52:54.207138 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81649) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 16:52:54.207350 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81649) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 16:52:54.207570 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81649) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 16:52:54.228719 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81650) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 16:52:54.229325 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81649) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 16:52:54.241938 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81650) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 16:52:54.242164 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81650) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 16:52:54.242374 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81649) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 16:52:54.242459 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81650) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 16:52:54.242589 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81650) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 16:52:54.252576 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81649) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 16:52:54.252664 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81650) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 16:52:54.252739 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81649) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 16:52:54.252802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81650) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 16:52:54.252860 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81649) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 16:52:54.252941 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81650) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 16:52:54.253016 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81649) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 16:52:54.253016 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81649) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 16:52:54.253016 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81649) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 16:52:54.268077 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.268197 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.268276 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 18 16:52:54.268348 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 18 16:52:54.268609 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.268691 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.268808 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 18 16:52:54.268897 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 18 16:52:54.269079 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.269159 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.269340 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.269591 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.269879 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.270022 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.270159 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.270294 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.279543 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.279822 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.279918 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.280097 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.280179 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.280368 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.280447 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.280656 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.287923 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81649) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 16:52:54.290516 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81650) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 16:52:54.290843 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81649) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 16:52:54.291108 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.291392 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.298707 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.298707 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.298707 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.298707 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.298707 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.298707 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.299202 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.299274 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81650) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 16:52:54.299346 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.299406 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.299406 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.299406 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.299406 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.299406 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.299406 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.299406 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.299794 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.299794 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.299794 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.317000 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.318456 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81649) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 16:52:54.318738 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.319715 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.321179 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81650) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 16:52:54.321464 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.323482 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81649) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 16:52:54.323767 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.324048 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.326251 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81650) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 16:52:54.326537 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.326912 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.330393 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81649) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 16:52:54.330680 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.334788 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81649) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 16:52:54.335094 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.335376 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.335656 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.335932 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.336206 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.336476 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.336742 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.340071 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81650) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 16:52:54.344400 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.344603 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81650) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 16:52:54.344897 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.345178 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.345723 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.345995 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.346268 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.346535 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.346797 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.354759 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.355069 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.359032 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81649) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 16:52:54.362286 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.362589 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.365434 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.365710 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.367393 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81649) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 16:52:54.369632 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81650) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 16:52:54.370521 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81649) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 16:52:54.372843 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.373139 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.373752 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.374060 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.374349 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.374635 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.374946 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.375230 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.377324 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81650) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 16:52:54.380471 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81650) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 16:52:54.383701 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.384000 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.384281 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.384561 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.384834 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.385107 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.390939 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81649) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 16:52:54.391242 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.391513 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.391780 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.398653 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.400214 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.400503 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.401068 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81650) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 16:52:54.401356 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.401624 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.401886 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.403156 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81649) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 16:52:54.403440 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.403721 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.404026 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.404305 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.408716 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.410775 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.411252 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.413978 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81650) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 16:52:54.414261 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.414542 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.414846 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.415161 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.416396 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81649) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 16:52:54.416683 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.416965 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.417249 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.417530 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.417808 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.418085 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.427039 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81650) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 16:52:54.427320 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.427595 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.427871 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.428146 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.428419 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.428702 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.433957 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81649) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 16:52:54.437006 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81649) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 16:52:54.439908 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.441338 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.443421 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81649) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 16:52:54.443707 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.444010 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.444256 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81650) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 16:52:54.447281 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81650) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 16:52:54.450149 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.450433 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.451608 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.451926 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81649) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 16:52:54.453072 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81650) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 16:52:54.453361 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.453653 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.454858 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.455267 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.455580 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.455860 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.460104 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.461559 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81650) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 16:52:54.463927 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81649) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 16:52:54.464221 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.464490 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.464573 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.464784 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.465061 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.465330 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.471605 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81649) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 16:52:54.471891 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.472175 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.472452 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.473229 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.473647 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.473930 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.474019 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81650) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 16:52:54.474200 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.474282 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.474479 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.474561 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.474758 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.475065 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.475332 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.475611 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.475871 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.481417 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81650) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 16:52:54.481652 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.481929 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.482209 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.482491 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.482762 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.483056 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.483325 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.483605 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.483881 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.484149 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.484408 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.484678 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.484936 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81650) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 16:52:54.510149 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55cc9635a9c0 pid: 81649 (default app) Aug 18 16:52:54.513555 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ba4ec-0362-41ec-8624-a161c766d453 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:52:54.513843 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ba4ec-0362-41ec-8624-a161c766d453 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:52:54.514116 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ba4ec-0362-41ec-8624-a161c766d453 None None] PATH_INFO: `/v3` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:52:54.518227 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1/1] 10.222.0.82 () {50 vars in 791 bytes} [Thu Aug 18 16:52:54 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:52:54.519196 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55cc9635a9c0 pid: 81650 (default app) Aug 18 16:53:01.821682 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ce6eb-8e94-4c7a-8b1f-7d73583b1464 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:01.822159 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ce6eb-8e94-4c7a-8b1f-7d73583b1464 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:01.822244 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ce6eb-8e94-4c7a-8b1f-7d73583b1464 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:01.823585 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 2/2] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:53:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:53:01.831007 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c55c4-b7fe-477a-a80c-20f526da6005 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:01.831396 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c55c4-b7fe-477a-a80c-20f526da6005 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:01.831558 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c55c4-b7fe-477a-a80c-20f526da6005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:01.901695 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: INFO dbcounter [None req-036c55c4-b7fe-477a-a80c-20f526da6005 None None] Registered counter for database keystone Aug 18 16:53:01.920928 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG oslo_db.sqlalchemy.engines [None req-036c55c4-b7fe-477a-a80c-20f526da6005 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81650) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 18 16:53:01.928043 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writer thread running {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 18 16:53:02.108782 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG passlib.handlers.bcrypt [None req-036c55c4-b7fe-477a-a80c-20f526da6005 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81650) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 18 16:53:02.109196 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG passlib.handlers.bcrypt [None req-036c55c4-b7fe-477a-a80c-20f526da6005 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81650) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 18 16:53:02.164138 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-036c55c4-b7fe-477a-a80c-20f526da6005 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:53:02.167218 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-036c55c4-b7fe-477a-a80c-20f526da6005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:53:02.293049 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1/3] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:53:01 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 465 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 18 16:53:02.734501 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5ce6eb-8e94-4c7a-8b1f-7d73583b1464 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:02.805377 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: INFO dbcounter [None req-bc5ce6eb-8e94-4c7a-8b1f-7d73583b1464 None None] Registered counter for database keystone Aug 18 16:53:02.826946 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG oslo_db.sqlalchemy.engines [None req-bc5ce6eb-8e94-4c7a-8b1f-7d73583b1464 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81649) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 18 16:53:02.833892 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writer thread running {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 18 16:53:02.953395 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5ce6eb-8e94-4c7a-8b1f-7d73583b1464 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:02.958442 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e4a8f0-d253-42be-9287-13b6c9929a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:02.959095 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e4a8f0-d253-42be-9287-13b6c9929a66 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:02.959375 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e4a8f0-d253-42be-9287-13b6c9929a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:02.959643 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e4a8f0-d253-42be-9287-13b6c9929a66 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:02.967533 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e4a8f0-d253-42be-9287-13b6c9929a66 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:03.018716 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: WARNING keystone.common.rbac_enforcer.enforcer [None req-10e4a8f0-d253-42be-9287-13b6c9929a66 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 18 16:53:03.019039 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e4a8f0-d253-42be-9287-13b6c9929a66 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:03.035712 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 3/4] 10.222.0.82 () {58 vars in 1324 bytes} [Thu Aug 18 16:53:02 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 16:53:03.041771 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036c55c4-b7fe-477a-a80c-20f526da6005 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:03.056555 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036c55c4-b7fe-477a-a80c-20f526da6005 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:03.060923 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbd2e69-dd27-4df3-b963-6d46454bb685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:03.062019 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbd2e69-dd27-4df3-b963-6d46454bb685 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:03.062303 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbd2e69-dd27-4df3-b963-6d46454bb685 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:03.062572 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbd2e69-dd27-4df3-b963-6d46454bb685 admin admin] PATH_INFO: `/v3/projects/77b6b270950a499cb6b0da7c630c03b6` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:03.064000 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbd2e69-dd27-4df3-b963-6d46454bb685 admin admin] RBAC: Authorizing `identity:get_project(project_id=77b6b270950a499cb6b0da7c630c03b6)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:03.117541 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: WARNING keystone.common.rbac_enforcer.enforcer [None req-4cbd2e69-dd27-4df3-b963-6d46454bb685 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 18 16:53:03.117882 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbd2e69-dd27-4df3-b963-6d46454bb685 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:03.122334 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 2/5] 10.222.0.82 () {56 vars in 1207 bytes} [Thu Aug 18 16:53:03 2022] GET /identity/v3/projects/77b6b270950a499cb6b0da7c630c03b6 => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:05.006827 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51257f53-b48a-4be4-a7c2-d22487bd3769 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:05.007357 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51257f53-b48a-4be4-a7c2-d22487bd3769 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:05.007467 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51257f53-b48a-4be4-a7c2-d22487bd3769 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:05.011452 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 4/6] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:53:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:53:05.016760 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cba3bb-3a1b-4c35-8cb2-9e398259ddd0 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:05.017114 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cba3bb-3a1b-4c35-8cb2-9e398259ddd0 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:05.017279 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cba3bb-3a1b-4c35-8cb2-9e398259ddd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:05.037626 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-b5cba3bb-3a1b-4c35-8cb2-9e398259ddd0 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:53:05.039169 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-b5cba3bb-3a1b-4c35-8cb2-9e398259ddd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:53:05.046980 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 3/7] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:53:05 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 18 16:53:05.318076 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51257f53-b48a-4be4-a7c2-d22487bd3769 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:05.330169 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51257f53-b48a-4be4-a7c2-d22487bd3769 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:05.339039 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ea7f19-cd7b-4d1b-a067-47fa1d6dc900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:05.339039 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ea7f19-cd7b-4d1b-a067-47fa1d6dc900 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:05.339039 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ea7f19-cd7b-4d1b-a067-47fa1d6dc900 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:05.339039 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ea7f19-cd7b-4d1b-a067-47fa1d6dc900 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:05.343353 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ea7f19-cd7b-4d1b-a067-47fa1d6dc900 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:05.345436 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ea7f19-cd7b-4d1b-a067-47fa1d6dc900 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:05.354112 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 5/8] 10.222.0.82 () {58 vars in 1324 bytes} [Thu Aug 18 16:53:05 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 16:53:05.359574 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cba3bb-3a1b-4c35-8cb2-9e398259ddd0 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:05.369379 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cba3bb-3a1b-4c35-8cb2-9e398259ddd0 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:05.373788 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59329295-92fe-4da4-847f-d81a49582b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:05.374416 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59329295-92fe-4da4-847f-d81a49582b8c admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:05.374702 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59329295-92fe-4da4-847f-d81a49582b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:05.375014 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59329295-92fe-4da4-847f-d81a49582b8c admin admin] PATH_INFO: `/v3/users/329c213a845c4ab2a0642e5a7dbd4b18` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:05.376332 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59329295-92fe-4da4-847f-d81a49582b8c admin admin] RBAC: Authorizing `identity:get_user(user_id=329c213a845c4ab2a0642e5a7dbd4b18)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:05.378190 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59329295-92fe-4da4-847f-d81a49582b8c admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:05.380326 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 4/9] 10.222.0.82 () {56 vars in 1198 bytes} [Thu Aug 18 16:53:05 2022] GET /identity/v3/users/329c213a845c4ab2a0642e5a7dbd4b18 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:07.327067 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da02b903-72a0-4d77-86a7-2db75425146c None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:07.327578 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da02b903-72a0-4d77-86a7-2db75425146c None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:07.327578 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da02b903-72a0-4d77-86a7-2db75425146c None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:07.328912 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 6/10] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:53:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:53:07.334768 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d912ab46-1625-4f32-8500-19e19f72053c None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:07.335181 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d912ab46-1625-4f32-8500-19e19f72053c None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:07.335476 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d912ab46-1625-4f32-8500-19e19f72053c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:07.353984 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-d912ab46-1625-4f32-8500-19e19f72053c None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:53:07.355202 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-d912ab46-1625-4f32-8500-19e19f72053c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:53:07.371267 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd1eeb-0e2d-42d8-804c-4faa439da41d None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:07.371555 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd1eeb-0e2d-42d8-804c-4faa439da41d None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:07.371998 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd1eeb-0e2d-42d8-804c-4faa439da41d None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:07.373323 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 7/11] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:53:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 16:53:07.378555 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da2fe3d-37b5-4ed5-b3e7-148011ab35e6 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:07.378844 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da2fe3d-37b5-4ed5-b3e7-148011ab35e6 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:07.379318 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da2fe3d-37b5-4ed5-b3e7-148011ab35e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:07.421177 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG passlib.handlers.bcrypt [None req-7da2fe3d-37b5-4ed5-b3e7-148011ab35e6 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81649) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 18 16:53:07.421605 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG passlib.handlers.bcrypt [None req-7da2fe3d-37b5-4ed5-b3e7-148011ab35e6 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81649) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 18 16:53:07.435036 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 5/12] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:53:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 16:53:07.468064 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-7da2fe3d-37b5-4ed5-b3e7-148011ab35e6 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:53:07.469650 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-7da2fe3d-37b5-4ed5-b3e7-148011ab35e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:53:07.566725 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 8/13] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:53:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 16:53:07.709195 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d912ab46-1625-4f32-8500-19e19f72053c None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:07.732502 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:07.811402 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d912ab46-1625-4f32-8500-19e19f72053c None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:07.832931 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da2fe3d-37b5-4ed5-b3e7-148011ab35e6 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:07.847658 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f2b660-a3c6-4ff1-8366-e89cb34a84b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:07.848306 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f2b660-a3c6-4ff1-8366-e89cb34a84b5 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:07.848591 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f2b660-a3c6-4ff1-8366-e89cb34a84b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:07.848871 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f2b660-a3c6-4ff1-8366-e89cb34a84b5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:07.859878 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f2b660-a3c6-4ff1-8366-e89cb34a84b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:53:07.868729 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-15f2b660-a3c6-4ff1-8366-e89cb34a84b5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 16:53:07.868729 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:53:07.868729 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:53:07.868729 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:53:07.868729 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:53:07.868729 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:53:07.868729 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:53:07.868729 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:53:07.868729 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:53:07.868729 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:53:07.868729 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:53:07.868729 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:53:07.868729 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:53:07.868729 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:53:07.868729 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:53:07.868729 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:53:07.869769 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:53:07.869769 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:53:07.869769 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:53:07.869769 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:53:07.869769 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:53:07.869769 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:53:07.869769 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:53:07.869769 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:53:07.869769 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:53:07.869769 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:53:07.869769 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:53:07.869769 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:53:07.869769 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:53:07.869769 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:53:07.869769 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:53:07.869769 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:53:07.875318 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:53:07.875318 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:53:07.875318 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:53:07.875318 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:53:07.875318 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:53:07.875318 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:53:07.875318 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:53:07.875318 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:53:07.875318 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:53:07.875318 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:53:07.875318 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 16:53:07.875318 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:53:07.875318 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 6/14] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 16:53:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:53:07.876066 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f2b660-a3c6-4ff1-8366-e89cb34a84b5 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:07.935732 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da2fe3d-37b5-4ed5-b3e7-148011ab35e6 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:07.938037 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:07.959140 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:07.973527 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639d2782-75a2-41b8-94fc-9c42b7a94b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:07.974341 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d2782-75a2-41b8-94fc-9c42b7a94b2e None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:07.974622 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d2782-75a2-41b8-94fc-9c42b7a94b2e None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:07.974909 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d2782-75a2-41b8-94fc-9c42b7a94b2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:07.976960 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f2b660-a3c6-4ff1-8366-e89cb34a84b5 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:08.010069 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8663f7-068c-4a76-b44c-7be4188202b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:08.010680 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8663f7-068c-4a76-b44c-7be4188202b6 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:08.010990 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8663f7-068c-4a76-b44c-7be4188202b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:08.011312 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8663f7-068c-4a76-b44c-7be4188202b6 None admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:08.012228 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8663f7-068c-4a76-b44c-7be4188202b6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:53:08.012588 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8663f7-068c-4a76-b44c-7be4188202b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:08.014362 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8663f7-068c-4a76-b44c-7be4188202b6 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:08.027231 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 7/15] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:53:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:08.032879 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8663f7-068c-4a76-b44c-7be4188202b6 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:08.041727 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639d2782-75a2-41b8-94fc-9c42b7a94b2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:08.043272 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639d2782-75a2-41b8-94fc-9c42b7a94b2e None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:08.132044 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8663f7-068c-4a76-b44c-7be4188202b6 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:08.138946 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 9/16] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:53:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 18 16:53:08.144455 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639d2782-75a2-41b8-94fc-9c42b7a94b2e None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:08.147159 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:08.166160 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a229aa-ce32-43f5-ab22-daf4e414c0b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:08.167016 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a229aa-ce32-43f5-ab22-daf4e414c0b2 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:08.167307 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a229aa-ce32-43f5-ab22-daf4e414c0b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:08.167580 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a229aa-ce32-43f5-ab22-daf4e414c0b2 None admin] PATH_INFO: `/v3/users/329c213a845c4ab2a0642e5a7dbd4b18` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:08.168717 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a229aa-ce32-43f5-ab22-daf4e414c0b2 None admin] RBAC: Authorizing `identity:get_user(user_id=329c213a845c4ab2a0642e5a7dbd4b18)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:08.170188 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a229aa-ce32-43f5-ab22-daf4e414c0b2 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:08.171780 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 8/17] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 16:53:08 2022] GET /identity/v3/users/329c213a845c4ab2a0642e5a7dbd4b18 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:53:08.178135 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a229aa-ce32-43f5-ab22-daf4e414c0b2 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:08.193588 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:08.241871 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639d2782-75a2-41b8-94fc-9c42b7a94b2e None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:08.275216 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aaf722-85a5-4b57-89c2-b7d41741d037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:08.276075 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aaf722-85a5-4b57-89c2-b7d41741d037 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:08.276358 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aaf722-85a5-4b57-89c2-b7d41741d037 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:08.276630 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aaf722-85a5-4b57-89c2-b7d41741d037 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:08.283322 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a229aa-ce32-43f5-ab22-daf4e414c0b2 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:08.294800 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aaf722-85a5-4b57-89c2-b7d41741d037 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:08.296304 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aaf722-85a5-4b57-89c2-b7d41741d037 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:08.314414 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-83aaf722-85a5-4b57-89c2-b7d41741d037 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:53:08.314414 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:53:08.314414 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:53:08.314414 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:53:08.314414 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:53:08.314414 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:53:08.314414 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:53:08.314414 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:53:08.314414 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:53:08.314414 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:53:08.314414 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:53:08.314414 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:53:08.314414 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:53:08.314414 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:53:08.314414 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:53:08.314414 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:53:08.315334 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:53:08.315334 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:53:08.315334 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:53:08.315334 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:53:08.315334 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:53:08.315334 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:53:08.315334 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:53:08.315334 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:53:08.315334 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:53:08.315334 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:53:08.315334 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:53:08.315334 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:53:08.315334 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:53:08.315334 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:53:08.315334 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:53:08.315334 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:53:08.315334 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:53:08.317263 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:53:08.317263 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:53:08.317263 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:53:08.317263 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 10/18] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:53:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 16:53:08.319320 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afbd4bc-ed45-45ed-abea-5ce79af6ba6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:08.319965 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afbd4bc-ed45-45ed-abea-5ce79af6ba6e None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:08.320249 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afbd4bc-ed45-45ed-abea-5ce79af6ba6e None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:08.320524 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afbd4bc-ed45-45ed-abea-5ce79af6ba6e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:08.321910 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afbd4bc-ed45-45ed-abea-5ce79af6ba6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:08.322037 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aaf722-85a5-4b57-89c2-b7d41741d037 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:08.323431 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afbd4bc-ed45-45ed-abea-5ce79af6ba6e None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:08.324979 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 9/19] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:53:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:53:08.330463 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afbd4bc-ed45-45ed-abea-5ce79af6ba6e None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:08.381226 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:08.411985 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:08.424320 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aaf722-85a5-4b57-89c2-b7d41741d037 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:08.432396 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afbd4bc-ed45-45ed-abea-5ce79af6ba6e None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:08.459065 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f835a3c2-cb85-4156-ad37-ff413d246b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:08.459679 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835a3c2-cb85-4156-ad37-ff413d246b4d None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:08.459958 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835a3c2-cb85-4156-ad37-ff413d246b4d None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:08.460235 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835a3c2-cb85-4156-ad37-ff413d246b4d None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:08.460879 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f835a3c2-cb85-4156-ad37-ff413d246b4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:53:08.461182 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f835a3c2-cb85-4156-ad37-ff413d246b4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:08.462602 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f835a3c2-cb85-4156-ad37-ff413d246b4d None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:08.466784 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3771a137-8467-49c6-982b-aeaa6e4d236f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:08.467420 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3771a137-8467-49c6-982b-aeaa6e4d236f None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:08.467706 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3771a137-8467-49c6-982b-aeaa6e4d236f None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:08.467984 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3771a137-8467-49c6-982b-aeaa6e4d236f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:08.468699 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3771a137-8467-49c6-982b-aeaa6e4d236f None admin] RBAC: Adding query filter params (role.id=2c6966cfb10f4d72b0242e9fb5e13b17, scope.domain.id=default, user.id=329c213a845c4ab2a0642e5a7dbd4b18) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:53:08.469008 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3771a137-8467-49c6-982b-aeaa6e4d236f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:08.470710 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3771a137-8467-49c6-982b-aeaa6e4d236f None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:08.478040 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 10/20] 10.222.0.82 () {56 vars in 1322 bytes} [Thu Aug 18 16:53:08 2022] GET /identity/v3/role_assignments?user.id=329c213a845c4ab2a0642e5a7dbd4b18&scope.domain.id=default&role.id=2c6966cfb10f4d72b0242e9fb5e13b17 => generated 233 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:08.481573 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 11/21] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:53:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:53:10.589905 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5802aeb7-a094-4dc6-b5f2-85c440e0428a None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:10.590443 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5802aeb7-a094-4dc6-b5f2-85c440e0428a None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:10.590443 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5802aeb7-a094-4dc6-b5f2-85c440e0428a None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:10.591991 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 11/22] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:53:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:53:10.608388 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea79b4ed-635c-43b8-8d78-1ff0d7cd2fdb None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:10.608858 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea79b4ed-635c-43b8-8d78-1ff0d7cd2fdb None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:10.608962 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea79b4ed-635c-43b8-8d78-1ff0d7cd2fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:10.632578 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-ea79b4ed-635c-43b8-8d78-1ff0d7cd2fdb None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:53:10.634009 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-ea79b4ed-635c-43b8-8d78-1ff0d7cd2fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:53:10.721164 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 12/23] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:53:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:53:10.755186 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c15ff1-1724-4cb0-87d3-872eda57a99b None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:10.755475 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c15ff1-1724-4cb0-87d3-872eda57a99b None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:10.755755 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c15ff1-1724-4cb0-87d3-872eda57a99b None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:10.757103 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 12/24] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:53:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 16:53:10.772753 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf42952-6df5-4c32-8fb9-6e4a109bfb62 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:10.773051 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf42952-6df5-4c32-8fb9-6e4a109bfb62 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:10.773336 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf42952-6df5-4c32-8fb9-6e4a109bfb62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:10.804580 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-faf42952-6df5-4c32-8fb9-6e4a109bfb62 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:53:10.805864 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-faf42952-6df5-4c32-8fb9-6e4a109bfb62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:53:10.851045 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:10.885158 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 13/25] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:53:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:53:11.015650 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d9195-baa8-46ba-9fcb-5ceae910018b None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:11.016353 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d9195-baa8-46ba-9fcb-5ceae910018b None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:11.016843 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d9195-baa8-46ba-9fcb-5ceae910018b None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:11.018457 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 13/26] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:53:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:53:11.024621 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da8738-4c6c-4930-9686-0fc39351da8b None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:11.025164 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da8738-4c6c-4930-9686-0fc39351da8b None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:11.025734 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da8738-4c6c-4930-9686-0fc39351da8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:11.053474 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-91da8738-4c6c-4930-9686-0fc39351da8b None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:53:11.055031 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-91da8738-4c6c-4930-9686-0fc39351da8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:53:11.095980 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915d9195-baa8-46ba-9fcb-5ceae910018b None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:11.171415 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 14/27] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:53:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:53:11.232834 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915d9195-baa8-46ba-9fcb-5ceae910018b None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:11.268437 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7a6277-146a-435b-a411-b8d10b2cbf3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:11.269521 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a6277-146a-435b-a411-b8d10b2cbf3b None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:11.270088 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a6277-146a-435b-a411-b8d10b2cbf3b None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:11.270569 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a6277-146a-435b-a411-b8d10b2cbf3b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:11.281856 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7a6277-146a-435b-a411-b8d10b2cbf3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:53:11.287386 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-2d7a6277-146a-435b-a411-b8d10b2cbf3b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 16:53:11.287386 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:53:11.287386 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:53:11.287386 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:53:11.287386 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:53:11.287386 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:53:11.287386 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:53:11.287386 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:53:11.287386 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:53:11.287386 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:53:11.287386 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:53:11.287386 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:53:11.287386 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:53:11.287386 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:53:11.287386 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:53:11.287386 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:53:11.288208 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:53:11.288208 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:53:11.288208 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:53:11.288208 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:53:11.288208 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:53:11.288208 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:53:11.288208 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:53:11.288208 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:53:11.288208 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:53:11.288208 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:53:11.288208 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:53:11.288208 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:53:11.288208 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:53:11.288208 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:53:11.288208 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:53:11.288208 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:53:11.288949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:53:11.288949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:53:11.288949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:53:11.288949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:53:11.288949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:53:11.288949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:53:11.288949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:53:11.288949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:53:11.288949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:53:11.288949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:53:11.288949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 16:53:11.288949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:53:11.291282 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 14/28] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 16:53:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 16:53:11.300730 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91da8738-4c6c-4930-9686-0fc39351da8b None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:11.423376 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91da8738-4c6c-4930-9686-0fc39351da8b None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:11.437348 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:11.442713 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae803767-8192-4b68-8d59-d869b134fadc None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:11.443031 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae803767-8192-4b68-8d59-d869b134fadc None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:11.443314 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae803767-8192-4b68-8d59-d869b134fadc None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:11.444788 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 15/29] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:53:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:53:11.454055 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f69808b-02c3-48fa-969e-d3db0c510655 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:11.454055 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f69808b-02c3-48fa-969e-d3db0c510655 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:11.454055 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f69808b-02c3-48fa-969e-d3db0c510655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:11.466064 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:11.474025 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-6f69808b-02c3-48fa-969e-d3db0c510655 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:53:11.476147 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-6f69808b-02c3-48fa-969e-d3db0c510655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:53:11.491801 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa69b5c3-5b39-4214-8940-940f86283456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:11.492434 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa69b5c3-5b39-4214-8940-940f86283456 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:11.492725 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa69b5c3-5b39-4214-8940-940f86283456 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:11.494535 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa69b5c3-5b39-4214-8940-940f86283456 None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:11.494535 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa69b5c3-5b39-4214-8940-940f86283456 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:53:11.494535 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa69b5c3-5b39-4214-8940-940f86283456 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:11.495567 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa69b5c3-5b39-4214-8940-940f86283456 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:11.517315 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 15/30] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:53:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:11.520028 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa69b5c3-5b39-4214-8940-940f86283456 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:11.585620 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 16/31] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:53:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 16:53:11.586592 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f69808b-02c3-48fa-969e-d3db0c510655 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:11.631572 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa69b5c3-5b39-4214-8940-940f86283456 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:11.716149 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72f949c-e027-4540-beec-5a4c59c4f727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:11.716726 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f949c-e027-4540-beec-5a4c59c4f727 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:11.717012 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f949c-e027-4540-beec-5a4c59c4f727 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:11.717291 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f949c-e027-4540-beec-5a4c59c4f727 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:11.730313 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f69808b-02c3-48fa-969e-d3db0c510655 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:11.742278 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72f949c-e027-4540-beec-5a4c59c4f727 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:11.743811 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72f949c-e027-4540-beec-5a4c59c4f727 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:11.765885 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-a72f949c-e027-4540-beec-5a4c59c4f727 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:53:11.765885 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:53:11.765885 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:53:11.765885 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:53:11.765885 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:53:11.765885 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:53:11.765885 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:53:11.765885 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:53:11.765885 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:53:11.765885 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:53:11.765885 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:53:11.765885 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:53:11.765885 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:53:11.765885 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:53:11.765885 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:53:11.765885 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:53:11.766754 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:53:11.766754 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:53:11.766754 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:53:11.766754 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:53:11.766754 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:53:11.766754 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:53:11.766754 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:53:11.766754 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:53:11.766754 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:53:11.766754 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:53:11.766754 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:53:11.766754 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:53:11.766754 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:53:11.766754 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:53:11.766754 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:53:11.766754 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:53:11.766754 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:53:11.767559 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:53:11.767559 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:53:11.767559 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:53:11.767559 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 16/32] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:53:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:53:11.768185 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72f949c-e027-4540-beec-5a4c59c4f727 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:11.796348 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:11.831629 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd7711b-ab00-4341-9868-0b6684370db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:11.832688 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd7711b-ab00-4341-9868-0b6684370db9 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:11.833145 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd7711b-ab00-4341-9868-0b6684370db9 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:11.833561 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd7711b-ab00-4341-9868-0b6684370db9 None admin] PATH_INFO: `/v3/users/329c213a845c4ab2a0642e5a7dbd4b18` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:11.834878 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd7711b-ab00-4341-9868-0b6684370db9 None admin] RBAC: Authorizing `identity:get_user(user_id=329c213a845c4ab2a0642e5a7dbd4b18)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:11.836569 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd7711b-ab00-4341-9868-0b6684370db9 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:11.838651 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 17/33] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 16:53:11 2022] GET /identity/v3/users/329c213a845c4ab2a0642e5a7dbd4b18 => generated 247 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:11.839603 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd7711b-ab00-4341-9868-0b6684370db9 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:11.849457 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:11.908816 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72f949c-e027-4540-beec-5a4c59c4f727 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:11.968168 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56113f04-c00d-477b-9dc2-8a20ce71b685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:11.969001 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56113f04-c00d-477b-9dc2-8a20ce71b685 None admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:11.969446 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56113f04-c00d-477b-9dc2-8a20ce71b685 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:11.971193 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56113f04-c00d-477b-9dc2-8a20ce71b685 None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:11.972040 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56113f04-c00d-477b-9dc2-8a20ce71b685 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:11.973565 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56113f04-c00d-477b-9dc2-8a20ce71b685 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:11.998096 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd7711b-ab00-4341-9868-0b6684370db9 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:12.033890 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e41c862-dd13-49de-9de3-74f3d40a462e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:12.034654 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e41c862-dd13-49de-9de3-74f3d40a462e None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:12.034988 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e41c862-dd13-49de-9de3-74f3d40a462e None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:12.035272 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e41c862-dd13-49de-9de3-74f3d40a462e None admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:12.035916 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e41c862-dd13-49de-9de3-74f3d40a462e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:53:12.036223 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e41c862-dd13-49de-9de3-74f3d40a462e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:12.037616 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e41c862-dd13-49de-9de3-74f3d40a462e None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:12.060791 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 18/34] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:53:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:12.060791 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e41c862-dd13-49de-9de3-74f3d40a462e None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:12.130994 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:12.173987 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e41c862-dd13-49de-9de3-74f3d40a462e None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:12.213242 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381c6e38-e902-46a5-baba-a7697c1703df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:12.214169 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c6e38-e902-46a5-baba-a7697c1703df None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:12.214739 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c6e38-e902-46a5-baba-a7697c1703df None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:12.215199 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c6e38-e902-46a5-baba-a7697c1703df None admin] PATH_INFO: `/v3/domains/default` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:12.216503 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381c6e38-e902-46a5-baba-a7697c1703df None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:12.218143 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381c6e38-e902-46a5-baba-a7697c1703df None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:12.220582 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 17/35] 10.222.0.82 () {60 vars in 1131 bytes} [Thu Aug 18 16:53:11 2022] POST /identity/v3/roles => generated 219 bytes in 453 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 16:53:12.220970 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 19/36] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:53:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:12.221770 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56113f04-c00d-477b-9dc2-8a20ce71b685 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:12.222171 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381c6e38-e902-46a5-baba-a7697c1703df None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:12.321859 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:12.343441 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56113f04-c00d-477b-9dc2-8a20ce71b685 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:12.377636 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381c6e38-e902-46a5-baba-a7697c1703df None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:12.398462 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84db8d16-6ae7-4215-ae18-85ea587d5e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:12.399579 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84db8d16-6ae7-4215-ae18-85ea587d5e81 None admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:12.399579 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84db8d16-6ae7-4215-ae18-85ea587d5e81 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:12.399807 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84db8d16-6ae7-4215-ae18-85ea587d5e81 None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:12.400488 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84db8d16-6ae7-4215-ae18-85ea587d5e81 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:12.402013 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84db8d16-6ae7-4215-ae18-85ea587d5e81 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:12.414279 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:12.421001 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab86a7c-e144-454a-b3fb-5f91b864e6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:12.421620 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab86a7c-e144-454a-b3fb-5f91b864e6fc None admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:12.426044 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab86a7c-e144-454a-b3fb-5f91b864e6fc None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:12.426044 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab86a7c-e144-454a-b3fb-5f91b864e6fc None admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:12.426044 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab86a7c-e144-454a-b3fb-5f91b864e6fc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:12.426044 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab86a7c-e144-454a-b3fb-5f91b864e6fc None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:12.498489 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 18/37] 10.222.0.82 () {60 vars in 1131 bytes} [Thu Aug 18 16:53:12 2022] POST /identity/v3/roles => generated 225 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 16:53:12.499891 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726b52fb-1a07-47ae-ab37-fbe30f602a4c None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:12.500246 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726b52fb-1a07-47ae-ab37-fbe30f602a4c None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:12.500609 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726b52fb-1a07-47ae-ab37-fbe30f602a4c None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:12.507023 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 19/38] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:53:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:53:12.507023 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726b52fb-1a07-47ae-ab37-fbe30f602a4c None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:12.630468 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726b52fb-1a07-47ae-ab37-fbe30f602a4c None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:12.670518 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9cce59-195f-4122-8fda-c476bc3caf57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:12.671270 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9cce59-195f-4122-8fda-c476bc3caf57 None admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:12.671552 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9cce59-195f-4122-8fda-c476bc3caf57 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:12.671827 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9cce59-195f-4122-8fda-c476bc3caf57 None admin] PATH_INFO: `/v3/domains/default/users/329c213a845c4ab2a0642e5a7dbd4b18/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:12.674410 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9cce59-195f-4122-8fda-c476bc3caf57 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=329c213a845c4ab2a0642e5a7dbd4b18, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:12.676578 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9cce59-195f-4122-8fda-c476bc3caf57 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:12.687709 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 20/39] 10.222.0.82 () {60 vars in 1141 bytes} [Thu Aug 18 16:53:12 2022] POST /identity/v3/projects => generated 301 bytes in 466 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 16:53:12.689177 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e4ec26-9c49-4a79-8731-4475b266f2c2 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:12.689470 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e4ec26-9c49-4a79-8731-4475b266f2c2 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:12.689746 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e4ec26-9c49-4a79-8731-4475b266f2c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:12.711883 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-34e4ec26-9c49-4a79-8731-4475b266f2c2 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:53:12.713482 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-34e4ec26-9c49-4a79-8731-4475b266f2c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:53:12.759157 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:12.765556 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 20/40] 10.222.0.82 () {58 vars in 1361 bytes} [Thu Aug 18 16:53:12 2022] PUT /identity/v3/domains/default/users/329c213a845c4ab2a0642e5a7dbd4b18/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 16:53:12.774182 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fa730c-41cb-41d6-b748-3a1301592f54 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:12.774182 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fa730c-41cb-41d6-b748-3a1301592f54 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:12.774182 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fa730c-41cb-41d6-b748-3a1301592f54 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:12.774730 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 21/41] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:53:12 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:53:12.782375 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c3451-ddc8-4566-ab7d-34c70826c218 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:12.782375 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c3451-ddc8-4566-ab7d-34c70826c218 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:12.782375 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c3451-ddc8-4566-ab7d-34c70826c218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:12.802738 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-cb1c3451-ddc8-4566-ab7d-34c70826c218 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:53:12.804561 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-cb1c3451-ddc8-4566-ab7d-34c70826c218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:53:12.813714 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 21/42] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:53:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:53:12.895693 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 22/43] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:53:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:53:13.086401 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e4ec26-9c49-4a79-8731-4475b266f2c2 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:13.166159 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1c3451-ddc8-4566-ab7d-34c70826c218 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:13.188130 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e4ec26-9c49-4a79-8731-4475b266f2c2 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:13.202232 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:13.225646 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a6542a-8ce5-43a2-82cc-92e73552ebaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:13.226634 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a6542a-8ce5-43a2-82cc-92e73552ebaf None admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:13.227197 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a6542a-8ce5-43a2-82cc-92e73552ebaf None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:13.239916 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a6542a-8ce5-43a2-82cc-92e73552ebaf None admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:13.239916 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a6542a-8ce5-43a2-82cc-92e73552ebaf None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:13.239916 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a6542a-8ce5-43a2-82cc-92e73552ebaf None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:13.273262 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1c3451-ddc8-4566-ab7d-34c70826c218 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:13.302684 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:13.308630 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e485fbee-4cfc-4f63-9eda-bf43ea4eaf7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:13.309322 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e485fbee-4cfc-4f63-9eda-bf43ea4eaf7f None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:13.320573 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e485fbee-4cfc-4f63-9eda-bf43ea4eaf7f None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:13.320573 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e485fbee-4cfc-4f63-9eda-bf43ea4eaf7f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:13.320573 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e485fbee-4cfc-4f63-9eda-bf43ea4eaf7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:13.320573 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e485fbee-4cfc-4f63-9eda-bf43ea4eaf7f None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:13.320573 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 23/44] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:53:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:53:13.320573 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e485fbee-4cfc-4f63-9eda-bf43ea4eaf7f None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:13.345017 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 22/45] 10.222.0.82 () {60 vars in 1131 bytes} [Thu Aug 18 16:53:13 2022] POST /identity/v3/roles => generated 223 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 16:53:13.429617 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e485fbee-4cfc-4f63-9eda-bf43ea4eaf7f None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:13.467688 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dae4075-cb53-4037-bbb5-358fba51399c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:13.468646 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae4075-cb53-4037-bbb5-358fba51399c None admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:13.469099 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae4075-cb53-4037-bbb5-358fba51399c None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:13.469796 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae4075-cb53-4037-bbb5-358fba51399c None admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:13.470620 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dae4075-cb53-4037-bbb5-358fba51399c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:13.473409 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dae4075-cb53-4037-bbb5-358fba51399c None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:13.570017 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 24/46] 10.222.0.82 () {60 vars in 1141 bytes} [Thu Aug 18 16:53:13 2022] POST /identity/v3/projects => generated 312 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 16:53:14.716889 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf48e74-ebee-49a8-8871-016ef96d9e9c None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:14.717450 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf48e74-ebee-49a8-8871-016ef96d9e9c None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:14.717450 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf48e74-ebee-49a8-8871-016ef96d9e9c None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:14.719002 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 23/47] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:53:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:53:14.724482 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fba4388-87fa-42d5-91b4-3195e90270aa None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:14.724872 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fba4388-87fa-42d5-91b4-3195e90270aa None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:14.725048 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fba4388-87fa-42d5-91b4-3195e90270aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:14.742894 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-9fba4388-87fa-42d5-91b4-3195e90270aa None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:53:14.744104 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-9fba4388-87fa-42d5-91b4-3195e90270aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:53:14.791397 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:14.811886 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 25/48] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:53:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:53:15.084251 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf48e74-ebee-49a8-8871-016ef96d9e9c None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:15.294520 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf48e74-ebee-49a8-8871-016ef96d9e9c None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:15.328253 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8c7e04-f0b6-4c1c-ba8b-50a14dbd7b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:15.328894 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8c7e04-f0b6-4c1c-ba8b-50a14dbd7b40 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:15.336423 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8c7e04-f0b6-4c1c-ba8b-50a14dbd7b40 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:15.336423 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8c7e04-f0b6-4c1c-ba8b-50a14dbd7b40 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:15.340246 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8c7e04-f0b6-4c1c-ba8b-50a14dbd7b40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:53:15.345405 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-8c8c7e04-f0b6-4c1c-ba8b-50a14dbd7b40 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 16:53:15.345405 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:53:15.345405 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:53:15.345405 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:53:15.345405 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:53:15.345405 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:53:15.345405 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:53:15.345405 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:53:15.345405 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:53:15.345405 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:53:15.345405 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:53:15.345405 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:53:15.345405 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:53:15.345405 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:53:15.345405 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:53:15.345405 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:53:15.346201 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:53:15.346201 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:53:15.346201 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:53:15.346201 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:53:15.346201 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:53:15.346201 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:53:15.346201 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:53:15.346201 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:53:15.346201 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:53:15.346201 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:53:15.346201 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:53:15.346201 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:53:15.346201 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:53:15.346201 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:53:15.346201 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:53:15.346201 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:53:15.347192 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:53:15.347192 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:53:15.347192 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:53:15.347192 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:53:15.347192 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:53:15.347192 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:53:15.347192 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:53:15.347192 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:53:15.347192 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:53:15.347192 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:53:15.347192 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 16:53:15.347192 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:53:15.347192 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 24/49] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 16:53:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:53:15.352309 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fba4388-87fa-42d5-91b4-3195e90270aa None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:15.478889 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec74fb1-1537-4baa-aae0-40e539558a0f None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:15.479210 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec74fb1-1537-4baa-aae0-40e539558a0f None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:15.479476 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec74fb1-1537-4baa-aae0-40e539558a0f None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:15.480864 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 25/50] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:53:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 16:53:15.486307 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfd7fb9-88cc-4394-b0c0-a31d312e7ca5 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:15.486593 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfd7fb9-88cc-4394-b0c0-a31d312e7ca5 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:15.486869 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfd7fb9-88cc-4394-b0c0-a31d312e7ca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:15.506225 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-7bfd7fb9-88cc-4394-b0c0-a31d312e7ca5 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:53:15.507486 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-7bfd7fb9-88cc-4394-b0c0-a31d312e7ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:53:15.533878 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:15.557276 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fba4388-87fa-42d5-91b4-3195e90270aa None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:15.575623 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 26/51] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:53:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 16:53:15.591410 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bca57c-b642-4ac3-8adb-7806b1be5194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:15.591962 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bca57c-b642-4ac3-8adb-7806b1be5194 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:15.592249 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bca57c-b642-4ac3-8adb-7806b1be5194 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:15.592526 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bca57c-b642-4ac3-8adb-7806b1be5194 None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:15.593183 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bca57c-b642-4ac3-8adb-7806b1be5194 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:53:15.593490 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bca57c-b642-4ac3-8adb-7806b1be5194 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:15.595208 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bca57c-b642-4ac3-8adb-7806b1be5194 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:15.603124 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 26/52] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:53:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:15.608965 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfd7fb9-88cc-4394-b0c0-a31d312e7ca5 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:15.706132 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfd7fb9-88cc-4394-b0c0-a31d312e7ca5 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:15.739541 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa357d7f-bf8b-42cb-99e5-5aabbb8309bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:15.740169 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa357d7f-bf8b-42cb-99e5-5aabbb8309bb None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:15.740452 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa357d7f-bf8b-42cb-99e5-5aabbb8309bb None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:15.740728 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa357d7f-bf8b-42cb-99e5-5aabbb8309bb None admin] PATH_INFO: `/v3/users/329c213a845c4ab2a0642e5a7dbd4b18` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:15.741866 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa357d7f-bf8b-42cb-99e5-5aabbb8309bb None admin] RBAC: Authorizing `identity:get_user(user_id=329c213a845c4ab2a0642e5a7dbd4b18)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:15.743374 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa357d7f-bf8b-42cb-99e5-5aabbb8309bb None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:15.744923 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 27/53] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 16:53:15 2022] GET /identity/v3/users/329c213a845c4ab2a0642e5a7dbd4b18 => generated 247 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:53:15.750450 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bca57c-b642-4ac3-8adb-7806b1be5194 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:15.797016 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:15.844146 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa357d7f-bf8b-42cb-99e5-5aabbb8309bb None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:15.853432 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bca57c-b642-4ac3-8adb-7806b1be5194 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:15.874116 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:15.887756 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580ceb6d-d673-4d9c-bb22-77f8cd0e9268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:15.888387 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580ceb6d-d673-4d9c-bb22-77f8cd0e9268 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:15.888673 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580ceb6d-d673-4d9c-bb22-77f8cd0e9268 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:15.888951 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580ceb6d-d673-4d9c-bb22-77f8cd0e9268 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:15.890053 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580ceb6d-d673-4d9c-bb22-77f8cd0e9268 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:15.891533 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580ceb6d-d673-4d9c-bb22-77f8cd0e9268 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:15.893146 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 27/54] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:53:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:15.899000 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580ceb6d-d673-4d9c-bb22-77f8cd0e9268 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:15.944825 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa357d7f-bf8b-42cb-99e5-5aabbb8309bb None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:15.978596 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3942272-cf42-4877-8b50-c845956184c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:15.979551 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3942272-cf42-4877-8b50-c845956184c0 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:15.979551 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3942272-cf42-4877-8b50-c845956184c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:15.979829 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3942272-cf42-4877-8b50-c845956184c0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:15.980937 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3942272-cf42-4877-8b50-c845956184c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:15.982405 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3942272-cf42-4877-8b50-c845956184c0 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:15.984173 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 28/55] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:53:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:53:15.990430 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3942272-cf42-4877-8b50-c845956184c0 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:15.997682 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580ceb6d-d673-4d9c-bb22-77f8cd0e9268 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:16.007785 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:16.030022 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbc6932-f035-4483-83dc-fc7ea9971c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:16.030628 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbc6932-f035-4483-83dc-fc7ea9971c1b None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:16.030925 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbc6932-f035-4483-83dc-fc7ea9971c1b None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:16.031231 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbc6932-f035-4483-83dc-fc7ea9971c1b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:16.031947 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbc6932-f035-4483-83dc-fc7ea9971c1b None admin] RBAC: Adding query filter params (role.id=2c6966cfb10f4d72b0242e9fb5e13b17, scope.domain.id=default, user.id=329c213a845c4ab2a0642e5a7dbd4b18) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:53:16.032253 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbc6932-f035-4483-83dc-fc7ea9971c1b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:16.033687 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbc6932-f035-4483-83dc-fc7ea9971c1b None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:16.043435 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 28/56] 10.222.0.82 () {56 vars in 1322 bytes} [Thu Aug 18 16:53:15 2022] GET /identity/v3/role_assignments?user.id=329c213a845c4ab2a0642e5a7dbd4b18&scope.domain.id=default&role.id=2c6966cfb10f4d72b0242e9fb5e13b17 => generated 530 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:53:16.088579 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:16.092736 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3942272-cf42-4877-8b50-c845956184c0 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:16.129827 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5842dca5-97a4-4eb3-b42e-2abbf9f7e7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:16.130365 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5842dca5-97a4-4eb3-b42e-2abbf9f7e7ba None admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:16.130771 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5842dca5-97a4-4eb3-b42e-2abbf9f7e7ba None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:16.131451 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5842dca5-97a4-4eb3-b42e-2abbf9f7e7ba None admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:16.132295 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5842dca5-97a4-4eb3-b42e-2abbf9f7e7ba None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:16.133943 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5842dca5-97a4-4eb3-b42e-2abbf9f7e7ba None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:16.242792 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 29/57] 10.222.0.82 () {60 vars in 1140 bytes} [Thu Aug 18 16:53:15 2022] POST /identity/v3/projects => generated 298 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 16:53:18.215377 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942f3e9d-bf24-44a9-afa2-e4e0406fc0ee None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:18.215915 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942f3e9d-bf24-44a9-afa2-e4e0406fc0ee None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:18.215915 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942f3e9d-bf24-44a9-afa2-e4e0406fc0ee None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:18.217280 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 29/58] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:53:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:53:18.222862 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e005d2bf-ef5c-48db-9468-ee5276852abc None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:18.223296 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e005d2bf-ef5c-48db-9468-ee5276852abc None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:18.223455 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e005d2bf-ef5c-48db-9468-ee5276852abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:18.241626 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-e005d2bf-ef5c-48db-9468-ee5276852abc None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:53:18.242824 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-e005d2bf-ef5c-48db-9468-ee5276852abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:53:18.308964 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 30/59] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:53:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:53:18.576710 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942f3e9d-bf24-44a9-afa2-e4e0406fc0ee None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:18.677245 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942f3e9d-bf24-44a9-afa2-e4e0406fc0ee None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:18.710180 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd58329a-0623-4960-a69d-95dff646fce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:18.710797 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd58329a-0623-4960-a69d-95dff646fce6 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:18.711111 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd58329a-0623-4960-a69d-95dff646fce6 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:18.711390 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd58329a-0623-4960-a69d-95dff646fce6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:18.712493 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd58329a-0623-4960-a69d-95dff646fce6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:18.713991 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd58329a-0623-4960-a69d-95dff646fce6 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:18.715560 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 30/60] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:53:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:18.722017 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e005d2bf-ef5c-48db-9468-ee5276852abc None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:18.802131 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:18.820315 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e005d2bf-ef5c-48db-9468-ee5276852abc None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:18.854528 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d2bc66-2acb-4354-a454-9bb28f0939b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:18.855120 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2bc66-2acb-4354-a454-9bb28f0939b6 None admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:18.855407 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2bc66-2acb-4354-a454-9bb28f0939b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:18.855683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2bc66-2acb-4354-a454-9bb28f0939b6 None admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:18.856337 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d2bc66-2acb-4354-a454-9bb28f0939b6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:18.857820 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d2bc66-2acb-4354-a454-9bb28f0939b6 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:18.967449 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 31/61] 10.222.0.82 () {60 vars in 1132 bytes} [Thu Aug 18 16:53:18 2022] POST /identity/v3/users => generated 275 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 16:53:21.896774 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee74b3ae-2979-4299-9c1a-94e7a1e8895c None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:21.897778 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee74b3ae-2979-4299-9c1a-94e7a1e8895c None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:21.898265 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee74b3ae-2979-4299-9c1a-94e7a1e8895c None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:21.899867 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 31/62] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:53:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:53:21.905377 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb221af-435d-4fd5-afb7-3e8db8fa77c3 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:21.906196 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb221af-435d-4fd5-afb7-3e8db8fa77c3 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:21.906682 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb221af-435d-4fd5-afb7-3e8db8fa77c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:21.935934 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-feb221af-435d-4fd5-afb7-3e8db8fa77c3 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:53:21.937485 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-feb221af-435d-4fd5-afb7-3e8db8fa77c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:53:22.032661 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 32/63] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:53:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:53:22.217070 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac9a366-b6c6-46fa-9fcf-1f773d2fd0c0 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:22.217734 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac9a366-b6c6-46fa-9fcf-1f773d2fd0c0 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:22.218219 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac9a366-b6c6-46fa-9fcf-1f773d2fd0c0 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:22.220323 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 32/64] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:53:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:53:22.225688 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d6a81f-33cd-487d-970f-049d93bb6a6c None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:22.225982 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d6a81f-33cd-487d-970f-049d93bb6a6c None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:22.226268 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d6a81f-33cd-487d-970f-049d93bb6a6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:22.247203 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-c8d6a81f-33cd-487d-970f-049d93bb6a6c None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:53:22.248397 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-c8d6a81f-33cd-487d-970f-049d93bb6a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:53:22.340500 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:22.349065 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 33/65] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:53:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:53:22.382271 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303acbd6-e835-4585-b782-8f505c7d372a None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:22.382443 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c5586c-b903-4626-9316-90319ae5e919 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:22.383000 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c5586c-b903-4626-9316-90319ae5e919 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:22.383000 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c5586c-b903-4626-9316-90319ae5e919 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:22.383261 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303acbd6-e835-4585-b782-8f505c7d372a None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:22.383703 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303acbd6-e835-4585-b782-8f505c7d372a None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:22.385013 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 34/66] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:53:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 16:53:22.385656 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 33/67] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:53:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:53:22.390787 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b3cdbe-857f-42b9-a5e7-a8f52ab6de6e None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:22.391028 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6971110-3cfa-46ba-a46a-34bfa6eecaa2 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:22.391287 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6971110-3cfa-46ba-a46a-34bfa6eecaa2 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:22.391570 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6971110-3cfa-46ba-a46a-34bfa6eecaa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:22.391925 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b3cdbe-857f-42b9-a5e7-a8f52ab6de6e None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:22.392396 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b3cdbe-857f-42b9-a5e7-a8f52ab6de6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:22.412382 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-b6971110-3cfa-46ba-a46a-34bfa6eecaa2 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:53:22.413820 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-b6971110-3cfa-46ba-a46a-34bfa6eecaa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:53:22.429426 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-c7b3cdbe-857f-42b9-a5e7-a8f52ab6de6e None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:53:22.430649 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-c7b3cdbe-857f-42b9-a5e7-a8f52ab6de6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:53:22.480388 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:22.502216 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 35/68] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:53:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 16:53:22.503525 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6971110-3cfa-46ba-a46a-34bfa6eecaa2 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:22.561655 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 34/69] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:53:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:53:22.595574 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dfc85f-021d-437a-a401-a4e766121ca7 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:22.595782 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dfc85f-021d-437a-a401-a4e766121ca7 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:22.596846 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dfc85f-021d-437a-a401-a4e766121ca7 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:22.598511 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 35/70] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:53:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 16:53:22.603536 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd692ac3-666a-413a-9593-495dc9dc05d1 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:22.604019 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd692ac3-666a-413a-9593-495dc9dc05d1 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:22.604468 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd692ac3-666a-413a-9593-495dc9dc05d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:22.622421 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-bd692ac3-666a-413a-9593-495dc9dc05d1 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:53:22.623941 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-bd692ac3-666a-413a-9593-495dc9dc05d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:53:22.655144 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6971110-3cfa-46ba-a46a-34bfa6eecaa2 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:22.711698 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:22.723783 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fdea6b-f297-4ae3-9782-0d66d140727a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:22.725303 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fdea6b-f297-4ae3-9782-0d66d140727a None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:22.725761 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fdea6b-f297-4ae3-9782-0d66d140727a None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:22.726177 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fdea6b-f297-4ae3-9782-0d66d140727a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:22.732651 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 36/71] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:53:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 16:53:22.733851 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd692ac3-666a-413a-9593-495dc9dc05d1 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:22.749404 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fdea6b-f297-4ae3-9782-0d66d140727a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:53:22.758188 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-24fdea6b-f297-4ae3-9782-0d66d140727a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 16:53:22.758188 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:53:22.758188 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:53:22.758188 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:53:22.758188 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:53:22.758188 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:53:22.758188 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:53:22.758188 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:53:22.758188 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:53:22.758188 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:53:22.758188 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:53:22.758188 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:53:22.758188 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:53:22.758188 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:53:22.758188 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:53:22.758188 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:53:22.759267 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:53:22.759267 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:53:22.759267 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:53:22.759267 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:53:22.759267 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:53:22.759267 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:53:22.759267 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:53:22.759267 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:53:22.759267 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:53:22.759267 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:53:22.759267 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:53:22.759267 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:53:22.759267 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:53:22.759267 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:53:22.759267 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:53:22.759267 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:53:22.760020 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:53:22.760020 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:53:22.760020 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:53:22.760020 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:53:22.760020 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:53:22.760020 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:53:22.760020 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:53:22.760020 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:53:22.760020 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:53:22.760020 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:53:22.760020 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 16:53:22.760020 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:53:22.762719 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 36/72] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:53:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:53:22.769524 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fdea6b-f297-4ae3-9782-0d66d140727a None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:22.886738 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd692ac3-666a-413a-9593-495dc9dc05d1 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:22.888691 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:22.892000 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fdea6b-f297-4ae3-9782-0d66d140727a None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:22.925163 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce47cc6b-d666-45b4-b71d-8ec962e11531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:22.925872 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce47cc6b-d666-45b4-b71d-8ec962e11531 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:22.926160 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce47cc6b-d666-45b4-b71d-8ec962e11531 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:22.926436 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce47cc6b-d666-45b4-b71d-8ec962e11531 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:22.926994 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379e8fb8-018a-4c89-9c39-1b35e603a68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:22.927611 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e8fb8-018a-4c89-9c39-1b35e603a68b None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:22.927897 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e8fb8-018a-4c89-9c39-1b35e603a68b None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:22.928184 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e8fb8-018a-4c89-9c39-1b35e603a68b None admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:22.928827 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379e8fb8-018a-4c89-9c39-1b35e603a68b None admin] RBAC: Adding query filter params (name=member) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:53:22.929135 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379e8fb8-018a-4c89-9c39-1b35e603a68b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:22.930544 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379e8fb8-018a-4c89-9c39-1b35e603a68b None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:22.938680 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 37/73] 10.222.0.82 () {56 vars in 1101 bytes} [Thu Aug 18 16:53:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:53:22.943221 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379e8fb8-018a-4c89-9c39-1b35e603a68b None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:22.944510 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce47cc6b-d666-45b4-b71d-8ec962e11531 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:53:22.950501 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-ce47cc6b-d666-45b4-b71d-8ec962e11531 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 16:53:22.950501 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:53:22.950501 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:53:22.950501 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:53:22.950501 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:53:22.950501 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:53:22.950501 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:53:22.950501 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:53:22.950501 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:53:22.950501 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:53:22.950501 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:53:22.950501 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:53:22.950501 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:53:22.950501 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:53:22.950501 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:53:22.950501 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:53:22.951375 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:53:22.951375 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:53:22.951375 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:53:22.951375 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:53:22.951375 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:53:22.951375 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:53:22.951375 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:53:22.951375 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:53:22.951375 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:53:22.951375 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:53:22.951375 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:53:22.951375 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:53:22.951375 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:53:22.951375 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:53:22.951375 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:53:22.951375 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:53:22.952110 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:53:22.952110 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:53:22.952110 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:53:22.952110 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:53:22.952110 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:53:22.952110 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:53:22.952110 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:53:22.952110 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:53:22.952110 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:53:22.952110 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:53:22.952110 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 16:53:22.952110 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:53:22.952110 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 37/74] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 16:53:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:53:22.952702 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce47cc6b-d666-45b4-b71d-8ec962e11531 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:22.998392 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:23.051685 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379e8fb8-018a-4c89-9c39-1b35e603a68b None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:23.061881 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce47cc6b-d666-45b4-b71d-8ec962e11531 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:23.085781 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c41c33-3b9e-4334-90c0-6952362eba39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:23.086827 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c41c33-3b9e-4334-90c0-6952362eba39 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:23.087389 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c41c33-3b9e-4334-90c0-6952362eba39 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:23.087912 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c41c33-3b9e-4334-90c0-6952362eba39 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:23.096195 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3710fc65-5491-4581-8ff6-ae149a22ae61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:23.096933 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3710fc65-5491-4581-8ff6-ae149a22ae61 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:23.097211 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3710fc65-5491-4581-8ff6-ae149a22ae61 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:23.097487 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3710fc65-5491-4581-8ff6-ae149a22ae61 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:23.099135 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c41c33-3b9e-4334-90c0-6952362eba39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:53:23.104258 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-29c41c33-3b9e-4334-90c0-6952362eba39 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 16:53:23.104258 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:53:23.104258 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:53:23.104258 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:53:23.104258 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:53:23.104258 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:53:23.104258 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:53:23.104258 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:53:23.104258 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:53:23.104258 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:53:23.104258 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:53:23.104258 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:53:23.104258 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:53:23.104258 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:53:23.104258 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:53:23.104258 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:53:23.105063 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:53:23.105063 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:53:23.105063 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:53:23.105063 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:53:23.105063 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:53:23.105063 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:53:23.105063 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:53:23.105063 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:53:23.105063 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:53:23.105063 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:53:23.105063 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:53:23.105063 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:53:23.105063 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:53:23.105063 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:53:23.105063 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:53:23.105063 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:53:23.105959 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:53:23.105959 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:53:23.105959 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:53:23.105959 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:53:23.105959 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:53:23.105959 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:53:23.105959 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:53:23.105959 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:53:23.105959 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:53:23.105959 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:53:23.105959 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 16:53:23.105959 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:53:23.105959 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 38/75] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:53:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:53:23.108106 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3710fc65-5491-4581-8ff6-ae149a22ae61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:53:23.110753 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c41c33-3b9e-4334-90c0-6952362eba39 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:23.113344 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-3710fc65-5491-4581-8ff6-ae149a22ae61 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 16:53:23.113344 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:53:23.113344 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:53:23.113344 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:53:23.113344 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:53:23.113344 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:53:23.113344 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:53:23.113344 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:53:23.113344 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:53:23.113344 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:53:23.113344 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:53:23.113344 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:53:23.113344 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:53:23.113344 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:53:23.113344 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:53:23.113344 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:53:23.114249 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:53:23.114249 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:53:23.114249 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:53:23.114249 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:53:23.114249 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:53:23.114249 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:53:23.114249 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:53:23.114249 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:53:23.114249 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:53:23.114249 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:53:23.114249 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:53:23.114249 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:53:23.114249 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:53:23.114249 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:53:23.114249 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:53:23.114249 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:53:23.115172 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:53:23.115172 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:53:23.115172 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:53:23.115172 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:53:23.115172 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:53:23.115172 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:53:23.115172 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:53:23.115172 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:53:23.115172 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:53:23.115172 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:53:23.115172 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 16:53:23.115172 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:53:23.115172 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 38/76] 10.222.0.82 () {56 vars in 1114 bytes} [Thu Aug 18 16:53:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:53:23.116105 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3710fc65-5491-4581-8ff6-ae149a22ae61 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:23.139516 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:23.215119 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3710fc65-5491-4581-8ff6-ae149a22ae61 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:23.215295 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c41c33-3b9e-4334-90c0-6952362eba39 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:23.224454 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:23.248929 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733d407b-06b7-4fc2-98b3-35a48f6243f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:23.249562 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733d407b-06b7-4fc2-98b3-35a48f6243f9 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:23.249658 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f77bbc7-e36d-4a5e-9151-dde8925ef72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:23.249846 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733d407b-06b7-4fc2-98b3-35a48f6243f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:23.250120 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733d407b-06b7-4fc2-98b3-35a48f6243f9 None admin] PATH_INFO: `/v3/users/3f70ad03afa246c393b5448fe7ee6860` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:23.250205 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77bbc7-e36d-4a5e-9151-dde8925ef72a None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:23.250435 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77bbc7-e36d-4a5e-9151-dde8925ef72a None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:23.250711 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77bbc7-e36d-4a5e-9151-dde8925ef72a None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:23.251402 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f77bbc7-e36d-4a5e-9151-dde8925ef72a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:53:23.251706 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f77bbc7-e36d-4a5e-9151-dde8925ef72a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:23.253799 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f77bbc7-e36d-4a5e-9151-dde8925ef72a None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:23.261819 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 39/77] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:53:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:23.262696 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f77bbc7-e36d-4a5e-9151-dde8925ef72a None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:23.265009 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733d407b-06b7-4fc2-98b3-35a48f6243f9 None admin] RBAC: Authorizing `identity:get_user(user_id=3f70ad03afa246c393b5448fe7ee6860)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:23.266862 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733d407b-06b7-4fc2-98b3-35a48f6243f9 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:23.268599 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 39/78] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 16:53:23 2022] GET /identity/v3/users/3f70ad03afa246c393b5448fe7ee6860 => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:23.269360 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733d407b-06b7-4fc2-98b3-35a48f6243f9 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:23.347581 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:23.363279 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f77bbc7-e36d-4a5e-9151-dde8925ef72a None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:23.367253 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733d407b-06b7-4fc2-98b3-35a48f6243f9 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:23.397592 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f77cca-5271-47b8-ad84-30b8e832e867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:23.398171 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f77cca-5271-47b8-ad84-30b8e832e867 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:23.398455 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f77cca-5271-47b8-ad84-30b8e832e867 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:23.398734 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f77cca-5271-47b8-ad84-30b8e832e867 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:23.399891 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f77cca-5271-47b8-ad84-30b8e832e867 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:23.401344 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f77cca-5271-47b8-ad84-30b8e832e867 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:23.401716 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fcb473-9e5d-4d40-9eb7-3d5dd58f406c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:23.402328 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fcb473-9e5d-4d40-9eb7-3d5dd58f406c None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:23.402614 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fcb473-9e5d-4d40-9eb7-3d5dd58f406c None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:23.402889 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fcb473-9e5d-4d40-9eb7-3d5dd58f406c None admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:23.403000 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 40/79] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:53:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:23.403675 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f77cca-5271-47b8-ad84-30b8e832e867 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:23.407693 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fcb473-9e5d-4d40-9eb7-3d5dd58f406c None admin] RBAC: Adding query filter params (name=member) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:53:23.408019 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fcb473-9e5d-4d40-9eb7-3d5dd58f406c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:23.409593 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fcb473-9e5d-4d40-9eb7-3d5dd58f406c None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:23.417853 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 40/80] 10.222.0.82 () {56 vars in 1101 bytes} [Thu Aug 18 16:53:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:23.418645 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fcb473-9e5d-4d40-9eb7-3d5dd58f406c None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:23.447497 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:23.504382 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f77cca-5271-47b8-ad84-30b8e832e867 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:23.519633 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fcb473-9e5d-4d40-9eb7-3d5dd58f406c None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:23.539653 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11575147-ef30-4442-86fa-66a226b5b89c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:23.540264 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11575147-ef30-4442-86fa-66a226b5b89c None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:23.540548 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11575147-ef30-4442-86fa-66a226b5b89c None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:23.540849 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11575147-ef30-4442-86fa-66a226b5b89c None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:23.541500 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11575147-ef30-4442-86fa-66a226b5b89c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:53:23.541810 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11575147-ef30-4442-86fa-66a226b5b89c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:23.543225 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11575147-ef30-4442-86fa-66a226b5b89c None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:23.551981 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 41/81] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 16:53:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:23.552780 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11575147-ef30-4442-86fa-66a226b5b89c None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:23.559579 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4da29d2-4082-479c-825d-20d40c7e2032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:23.560221 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da29d2-4082-479c-825d-20d40c7e2032 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:23.560510 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da29d2-4082-479c-825d-20d40c7e2032 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:23.560791 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da29d2-4082-479c-825d-20d40c7e2032 None admin] PATH_INFO: `/v3/users/329c213a845c4ab2a0642e5a7dbd4b18` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:23.561922 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4da29d2-4082-479c-825d-20d40c7e2032 None admin] RBAC: Authorizing `identity:get_user(user_id=329c213a845c4ab2a0642e5a7dbd4b18)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:23.563436 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4da29d2-4082-479c-825d-20d40c7e2032 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:23.565369 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 41/82] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 16:53:23 2022] GET /identity/v3/users/329c213a845c4ab2a0642e5a7dbd4b18 => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:23.566097 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4da29d2-4082-479c-825d-20d40c7e2032 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:23.570207 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:23.653206 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11575147-ef30-4442-86fa-66a226b5b89c None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:23.661291 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:23.666956 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4da29d2-4082-479c-825d-20d40c7e2032 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:23.687544 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238321c9-0521-40c7-b147-00053a9eb0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:23.688153 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238321c9-0521-40c7-b147-00053a9eb0cb None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:23.688434 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238321c9-0521-40c7-b147-00053a9eb0cb None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:23.688713 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238321c9-0521-40c7-b147-00053a9eb0cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:23.701226 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e1189a-534d-45ed-bbdd-91bf03dc72d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:23.701837 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1189a-534d-45ed-bbdd-91bf03dc72d6 None admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:23.702121 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1189a-534d-45ed-bbdd-91bf03dc72d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:23.702399 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1189a-534d-45ed-bbdd-91bf03dc72d6 None admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:23.703071 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e1189a-534d-45ed-bbdd-91bf03dc72d6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:23.704529 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e1189a-534d-45ed-bbdd-91bf03dc72d6 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:23.754064 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238321c9-0521-40c7-b147-00053a9eb0cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:23.755558 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238321c9-0521-40c7-b147-00053a9eb0cb None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:23.769456 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:23.772240 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 42/83] 10.222.0.82 () {60 vars in 1132 bytes} [Thu Aug 18 16:53:23 2022] POST /identity/v3/users => generated 289 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 16:53:23.772873 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e1189a-534d-45ed-bbdd-91bf03dc72d6 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:23.887715 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e1189a-534d-45ed-bbdd-91bf03dc72d6 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:23.894210 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 42/84] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:53:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 16:53:23.894210 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238321c9-0521-40c7-b147-00053a9eb0cb None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:23.937045 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1806403f-2598-4241-8473-d46e736179fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:23.937696 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1806403f-2598-4241-8473-d46e736179fd None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:23.938617 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1806403f-2598-4241-8473-d46e736179fd None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:23.938617 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1806403f-2598-4241-8473-d46e736179fd None admin] PATH_INFO: `/v3/users/3f70ad03afa246c393b5448fe7ee6860` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:23.939577 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1806403f-2598-4241-8473-d46e736179fd None admin] RBAC: Authorizing `identity:get_user(user_id=3f70ad03afa246c393b5448fe7ee6860)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:23.941101 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1806403f-2598-4241-8473-d46e736179fd None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:23.943831 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 43/85] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 16:53:23 2022] GET /identity/v3/users/3f70ad03afa246c393b5448fe7ee6860 => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:23.943831 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1806403f-2598-4241-8473-d46e736179fd None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:23.968146 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:24.013558 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238321c9-0521-40c7-b147-00053a9eb0cb None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:24.030678 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:24.059572 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07fe03b-396d-4077-910e-bd70276f4204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:24.060218 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07fe03b-396d-4077-910e-bd70276f4204 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:24.060500 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07fe03b-396d-4077-910e-bd70276f4204 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:24.060778 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07fe03b-396d-4077-910e-bd70276f4204 None admin] PATH_INFO: `/v3/users/3f70ad03afa246c393b5448fe7ee6860` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:24.062151 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07fe03b-396d-4077-910e-bd70276f4204 None admin] RBAC: Authorizing `identity:get_user(user_id=3f70ad03afa246c393b5448fe7ee6860)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:24.063773 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07fe03b-396d-4077-910e-bd70276f4204 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:24.066285 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 43/86] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 16:53:23 2022] GET /identity/v3/users/3f70ad03afa246c393b5448fe7ee6860 => generated 275 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:24.069533 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1806403f-2598-4241-8473-d46e736179fd None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:24.071311 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07fe03b-396d-4077-910e-bd70276f4204 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:24.116719 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66ba9bf-7223-41cd-b516-b5583e2f9b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:24.117341 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ba9bf-7223-41cd-b516-b5583e2f9b96 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:24.117631 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ba9bf-7223-41cd-b516-b5583e2f9b96 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:24.117898 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ba9bf-7223-41cd-b516-b5583e2f9b96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:24.197041 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66ba9bf-7223-41cd-b516-b5583e2f9b96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:24.198455 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66ba9bf-7223-41cd-b516-b5583e2f9b96 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:24.204771 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07fe03b-396d-4077-910e-bd70276f4204 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:24.221698 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:24.251259 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc2451d-e648-4f4a-a5cb-fc11e8154f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:24.251919 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc2451d-e648-4f4a-a5cb-fc11e8154f10 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:24.252755 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc2451d-e648-4f4a-a5cb-fc11e8154f10 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:24.253177 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc2451d-e648-4f4a-a5cb-fc11e8154f10 None admin] PATH_INFO: `/v3/projects/62beed06c75a4725ae7834995533d0a0` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:24.254983 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc2451d-e648-4f4a-a5cb-fc11e8154f10 None admin] RBAC: Authorizing `identity:get_project(project_id=62beed06c75a4725ae7834995533d0a0)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:24.256987 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc2451d-e648-4f4a-a5cb-fc11e8154f10 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:24.259746 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 44/87] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 16:53:24 2022] GET /identity/v3/projects/62beed06c75a4725ae7834995533d0a0 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:24.263357 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc2451d-e648-4f4a-a5cb-fc11e8154f10 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:24.317908 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:24.332744 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 44/88] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:53:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 16:53:24.333448 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66ba9bf-7223-41cd-b516-b5583e2f9b96 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:24.373194 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc2451d-e648-4f4a-a5cb-fc11e8154f10 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:24.435349 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba8005f-4f89-4a35-8d20-867782b9a1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:24.436615 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba8005f-4f89-4a35-8d20-867782b9a1ac None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:24.436999 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba8005f-4f89-4a35-8d20-867782b9a1ac None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:24.437640 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba8005f-4f89-4a35-8d20-867782b9a1ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:24.462644 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66ba9bf-7223-41cd-b516-b5583e2f9b96 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:24.487830 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:24.507493 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb60c8c-e7ec-47b8-a67d-f6438ef19083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:24.508204 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb60c8c-e7ec-47b8-a67d-f6438ef19083 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:24.508577 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb60c8c-e7ec-47b8-a67d-f6438ef19083 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:24.508856 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb60c8c-e7ec-47b8-a67d-f6438ef19083 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:24.522793 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba8005f-4f89-4a35-8d20-867782b9a1ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:24.524459 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba8005f-4f89-4a35-8d20-867782b9a1ac None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:24.575856 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:24.585443 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb60c8c-e7ec-47b8-a67d-f6438ef19083 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:24.587220 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb60c8c-e7ec-47b8-a67d-f6438ef19083 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:24.633634 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 45/89] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:53:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 16:53:24.633634 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba8005f-4f89-4a35-8d20-867782b9a1ac None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:24.701141 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 45/90] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:53:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 16:53:24.702230 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb60c8c-e7ec-47b8-a67d-f6438ef19083 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:24.723781 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:24.731352 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba8005f-4f89-4a35-8d20-867782b9a1ac None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:24.768925 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96448bd7-2a2d-4c6e-bce9-bae583d23e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:24.769538 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96448bd7-2a2d-4c6e-bce9-bae583d23e76 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:24.769825 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96448bd7-2a2d-4c6e-bce9-bae583d23e76 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:24.770100 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96448bd7-2a2d-4c6e-bce9-bae583d23e76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:24.770836 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96448bd7-2a2d-4c6e-bce9-bae583d23e76 None admin] RBAC: Adding query filter params (role.id=36d6d323d9e64e659591dbaea3081bcc, scope.project.id=62beed06c75a4725ae7834995533d0a0, user.id=3f70ad03afa246c393b5448fe7ee6860) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:53:24.771181 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96448bd7-2a2d-4c6e-bce9-bae583d23e76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:24.773084 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96448bd7-2a2d-4c6e-bce9-bae583d23e76 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:24.781047 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 46/91] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 16:53:24 2022] GET /identity/v3/role_assignments?user.id=3f70ad03afa246c393b5448fe7ee6860&scope.project.id=62beed06c75a4725ae7834995533d0a0&role.id=36d6d323d9e64e659591dbaea3081bcc => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:24.781653 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96448bd7-2a2d-4c6e-bce9-bae583d23e76 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:24.816048 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:24.842837 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb60c8c-e7ec-47b8-a67d-f6438ef19083 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:24.914075 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962bb87d-c457-42a1-a7ee-1014c06ffa5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:24.914735 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962bb87d-c457-42a1-a7ee-1014c06ffa5c None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:24.915053 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962bb87d-c457-42a1-a7ee-1014c06ffa5c None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:24.915333 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962bb87d-c457-42a1-a7ee-1014c06ffa5c None admin] PATH_INFO: `/v3/projects/62beed06c75a4725ae7834995533d0a0` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:24.917475 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962bb87d-c457-42a1-a7ee-1014c06ffa5c None admin] RBAC: Authorizing `identity:get_project(project_id=62beed06c75a4725ae7834995533d0a0)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:24.918993 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962bb87d-c457-42a1-a7ee-1014c06ffa5c None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:24.920868 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 46/92] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 16:53:24 2022] GET /identity/v3/projects/62beed06c75a4725ae7834995533d0a0 => generated 298 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:24.921474 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962bb87d-c457-42a1-a7ee-1014c06ffa5c None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:24.982548 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96448bd7-2a2d-4c6e-bce9-bae583d23e76 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:25.056468 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66afd23-f004-4b56-8010-6df4e0a1ea18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:25.057800 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66afd23-f004-4b56-8010-6df4e0a1ea18 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:25.058089 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66afd23-f004-4b56-8010-6df4e0a1ea18 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:25.058368 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66afd23-f004-4b56-8010-6df4e0a1ea18 None admin] PATH_INFO: `/v3/projects/244b82f9ef034b6bb3d250becacf818d` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:25.060493 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66afd23-f004-4b56-8010-6df4e0a1ea18 None admin] RBAC: Authorizing `identity:get_project(project_id=244b82f9ef034b6bb3d250becacf818d)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:25.061980 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66afd23-f004-4b56-8010-6df4e0a1ea18 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:25.063660 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 47/93] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 16:53:24 2022] GET /identity/v3/projects/244b82f9ef034b6bb3d250becacf818d => generated 312 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:25.064434 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66afd23-f004-4b56-8010-6df4e0a1ea18 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:25.105101 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:25.128204 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962bb87d-c457-42a1-a7ee-1014c06ffa5c None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:25.195382 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcfbc53-ea60-4a2c-91e5-4cc1697927e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:25.196035 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcfbc53-ea60-4a2c-91e5-4cc1697927e9 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:25.196323 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcfbc53-ea60-4a2c-91e5-4cc1697927e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:25.196604 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcfbc53-ea60-4a2c-91e5-4cc1697927e9 None admin] PATH_INFO: `/v3/projects/62beed06c75a4725ae7834995533d0a0` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:25.203627 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcfbc53-ea60-4a2c-91e5-4cc1697927e9 None admin] RBAC: Authorizing `identity:get_project(project_id=62beed06c75a4725ae7834995533d0a0)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:25.205114 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcfbc53-ea60-4a2c-91e5-4cc1697927e9 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:25.207474 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 47/94] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 16:53:24 2022] GET /identity/v3/projects/62beed06c75a4725ae7834995533d0a0 => generated 298 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:25.208043 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcfbc53-ea60-4a2c-91e5-4cc1697927e9 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:25.222545 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:25.250327 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66afd23-f004-4b56-8010-6df4e0a1ea18 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:25.280740 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672d1795-2f7d-42b0-8e66-9638c3416aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:25.281355 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d1795-2f7d-42b0-8e66-9638c3416aef None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:25.281641 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d1795-2f7d-42b0-8e66-9638c3416aef None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:25.281925 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d1795-2f7d-42b0-8e66-9638c3416aef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:25.282659 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672d1795-2f7d-42b0-8e66-9638c3416aef None admin] RBAC: Adding query filter params (role.id=2c6966cfb10f4d72b0242e9fb5e13b17, scope.project.id=62beed06c75a4725ae7834995533d0a0, user.id=329c213a845c4ab2a0642e5a7dbd4b18) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:53:25.282994 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672d1795-2f7d-42b0-8e66-9638c3416aef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:25.285891 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672d1795-2f7d-42b0-8e66-9638c3416aef None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:25.297680 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 48/95] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 16:53:25 2022] GET /identity/v3/role_assignments?user.id=329c213a845c4ab2a0642e5a7dbd4b18&scope.project.id=62beed06c75a4725ae7834995533d0a0&role.id=2c6966cfb10f4d72b0242e9fb5e13b17 => generated 259 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:25.298509 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672d1795-2f7d-42b0-8e66-9638c3416aef None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:25.433010 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcfbc53-ea60-4a2c-91e5-4cc1697927e9 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:25.505394 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fc0b98-bf94-4284-95f6-ff1dbf55245e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:25.505987 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc0b98-bf94-4284-95f6-ff1dbf55245e None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:25.506270 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc0b98-bf94-4284-95f6-ff1dbf55245e None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:25.506586 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc0b98-bf94-4284-95f6-ff1dbf55245e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:25.507353 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fc0b98-bf94-4284-95f6-ff1dbf55245e None admin] RBAC: Adding query filter params (role.id=36d6d323d9e64e659591dbaea3081bcc, scope.project.id=244b82f9ef034b6bb3d250becacf818d, user.id=3f70ad03afa246c393b5448fe7ee6860) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:53:25.507894 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fc0b98-bf94-4284-95f6-ff1dbf55245e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:25.509979 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fc0b98-bf94-4284-95f6-ff1dbf55245e None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:25.513682 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672d1795-2f7d-42b0-8e66-9638c3416aef None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:25.525646 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 48/96] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 16:53:25 2022] GET /identity/v3/role_assignments?user.id=3f70ad03afa246c393b5448fe7ee6860&scope.project.id=244b82f9ef034b6bb3d250becacf818d&role.id=36d6d323d9e64e659591dbaea3081bcc => generated 259 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:25.558605 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142a0855-8b19-4445-9b39-cc804b3ec021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:25.559265 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a0855-8b19-4445-9b39-cc804b3ec021 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:25.559545 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a0855-8b19-4445-9b39-cc804b3ec021 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:25.559819 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a0855-8b19-4445-9b39-cc804b3ec021 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:25.560569 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142a0855-8b19-4445-9b39-cc804b3ec021 None admin] RBAC: Adding query filter params (role.id=f41f6f29d9de4b1985c8c8c14f002cf2, scope.project.id=62beed06c75a4725ae7834995533d0a0, user.id=3f70ad03afa246c393b5448fe7ee6860) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:53:25.560874 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142a0855-8b19-4445-9b39-cc804b3ec021 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:25.562368 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142a0855-8b19-4445-9b39-cc804b3ec021 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:25.569197 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 49/97] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 16:53:25 2022] GET /identity/v3/role_assignments?user.id=3f70ad03afa246c393b5448fe7ee6860&scope.project.id=62beed06c75a4725ae7834995533d0a0&role.id=f41f6f29d9de4b1985c8c8c14f002cf2 => generated 259 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:26.054564 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e59e089-e8e0-46fd-9a89-ca33337843cb None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:26.054846 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e59e089-e8e0-46fd-9a89-ca33337843cb None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:26.055392 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e59e089-e8e0-46fd-9a89-ca33337843cb None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:26.057185 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 49/98] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:53:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:53:26.058843 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c34fcf1-333c-48c2-907f-26b0641f656c None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:26.059739 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c34fcf1-333c-48c2-907f-26b0641f656c None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:26.060206 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c34fcf1-333c-48c2-907f-26b0641f656c None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:26.061710 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 50/99] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:53:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:53:26.067771 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd59b48-002a-4e7a-ab8a-4483b105ebe9 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:26.068308 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd59b48-002a-4e7a-ab8a-4483b105ebe9 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:26.068776 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd59b48-002a-4e7a-ab8a-4483b105ebe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:26.078122 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9823e0-b8dd-41ef-b755-ec202650e8a4 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:26.083885 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9823e0-b8dd-41ef-b755-ec202650e8a4 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:26.084488 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9823e0-b8dd-41ef-b755-ec202650e8a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:26.096656 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-ddd59b48-002a-4e7a-ab8a-4483b105ebe9 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:53:26.099255 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-ddd59b48-002a-4e7a-ab8a-4483b105ebe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:53:26.119116 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-fc9823e0-b8dd-41ef-b755-ec202650e8a4 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:53:26.120684 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-fc9823e0-b8dd-41ef-b755-ec202650e8a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:53:26.125151 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:26.133898 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:26.232753 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 50/100] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:53:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:53:26.236821 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 51/101] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:53:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 16:53:26.552857 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd59b48-002a-4e7a-ab8a-4483b105ebe9 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:26.683097 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd59b48-002a-4e7a-ab8a-4483b105ebe9 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:26.719754 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d723b155-9856-4158-aff2-bdd196e8dd52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:26.720398 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d723b155-9856-4158-aff2-bdd196e8dd52 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:26.720678 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d723b155-9856-4158-aff2-bdd196e8dd52 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:26.721145 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d723b155-9856-4158-aff2-bdd196e8dd52 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:26.732112 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d723b155-9856-4158-aff2-bdd196e8dd52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:53:26.737184 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-d723b155-9856-4158-aff2-bdd196e8dd52 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 16:53:26.737184 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:53:26.737184 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:53:26.737184 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:53:26.737184 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:53:26.737184 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:53:26.737184 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:53:26.737184 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:53:26.737184 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:53:26.737184 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:53:26.737184 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:53:26.737184 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:53:26.737184 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:53:26.737184 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:53:26.737184 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:53:26.737184 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:53:26.738166 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:53:26.738166 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:53:26.738166 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:53:26.738166 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:53:26.738166 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:53:26.738166 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:53:26.738166 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:53:26.738166 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:53:26.738166 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:53:26.738166 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:53:26.738166 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:53:26.738166 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:53:26.738166 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:53:26.738166 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:53:26.738166 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:53:26.738166 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:53:26.739131 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:53:26.739131 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:53:26.739131 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:53:26.739131 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:53:26.739131 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:53:26.739131 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:53:26.739131 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:53:26.739131 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:53:26.739131 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:53:26.739131 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:53:26.739131 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 16:53:26.739131 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:53:26.739131 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 51/102] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:53:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:53:26.746671 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9823e0-b8dd-41ef-b755-ec202650e8a4 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:26.808301 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d723b155-9856-4158-aff2-bdd196e8dd52 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:26.828571 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:26.942076 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9823e0-b8dd-41ef-b755-ec202650e8a4 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:26.984419 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d723b155-9856-4158-aff2-bdd196e8dd52 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:27.009605 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d828c2-00e4-4d96-bf6d-e1587a0ec412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:27.010587 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d828c2-00e4-4d96-bf6d-e1587a0ec412 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:27.011127 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d828c2-00e4-4d96-bf6d-e1587a0ec412 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:27.011603 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d828c2-00e4-4d96-bf6d-e1587a0ec412 None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:27.012482 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d828c2-00e4-4d96-bf6d-e1587a0ec412 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:53:27.013154 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d828c2-00e4-4d96-bf6d-e1587a0ec412 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:27.015976 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d828c2-00e4-4d96-bf6d-e1587a0ec412 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:27.031823 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 52/103] 10.222.0.82 () {56 vars in 1101 bytes} [Thu Aug 18 16:53:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:27.040074 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d828c2-00e4-4d96-bf6d-e1587a0ec412 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:27.051330 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:27.075796 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cd1eb9-9c53-45d0-8f5c-1f79ecfe3c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:27.076842 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cd1eb9-9c53-45d0-8f5c-1f79ecfe3c17 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:27.077346 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cd1eb9-9c53-45d0-8f5c-1f79ecfe3c17 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:27.077822 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cd1eb9-9c53-45d0-8f5c-1f79ecfe3c17 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:27.079481 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cd1eb9-9c53-45d0-8f5c-1f79ecfe3c17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:27.081528 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cd1eb9-9c53-45d0-8f5c-1f79ecfe3c17 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:27.083388 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 52/104] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:53:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:53:27.089820 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cd1eb9-9c53-45d0-8f5c-1f79ecfe3c17 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:27.230691 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d828c2-00e4-4d96-bf6d-e1587a0ec412 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:27.256983 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:27.303387 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cd1eb9-9c53-45d0-8f5c-1f79ecfe3c17 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:27.323632 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0356289-c164-4656-9d0c-310bc968cc66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:27.324287 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0356289-c164-4656-9d0c-310bc968cc66 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:27.324570 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0356289-c164-4656-9d0c-310bc968cc66 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:27.324854 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0356289-c164-4656-9d0c-310bc968cc66 None admin] PATH_INFO: `/v3/users/f601719176bc414ab763d3f1fd6662d8` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:27.355748 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0356289-c164-4656-9d0c-310bc968cc66 None admin] RBAC: Authorizing `identity:get_user(user_id=f601719176bc414ab763d3f1fd6662d8)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:27.357119 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0356289-c164-4656-9d0c-310bc968cc66 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:27.359317 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 53/105] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 16:53:27 2022] GET /identity/v3/users/f601719176bc414ab763d3f1fd6662d8 => generated 289 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:27.368346 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0356289-c164-4656-9d0c-310bc968cc66 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:27.371230 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b83bbd-f101-437a-a257-eaae5ed74888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:27.371865 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b83bbd-f101-437a-a257-eaae5ed74888 None admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:27.372132 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b83bbd-f101-437a-a257-eaae5ed74888 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:27.372408 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b83bbd-f101-437a-a257-eaae5ed74888 None admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:27.378646 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b83bbd-f101-437a-a257-eaae5ed74888 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:27.380172 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b83bbd-f101-437a-a257-eaae5ed74888 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:27.468323 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:27.478287 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 53/106] 10.222.0.82 () {60 vars in 1141 bytes} [Thu Aug 18 16:53:27 2022] POST /identity/v3/projects => generated 302 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 16:53:27.525472 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0356289-c164-4656-9d0c-310bc968cc66 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:27.572081 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9937fee9-a8bb-43da-91c5-2cb8e7e7dab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:27.572930 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937fee9-a8bb-43da-91c5-2cb8e7e7dab8 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:27.573350 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937fee9-a8bb-43da-91c5-2cb8e7e7dab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:27.573757 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937fee9-a8bb-43da-91c5-2cb8e7e7dab8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:27.670246 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9937fee9-a8bb-43da-91c5-2cb8e7e7dab8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:27.672049 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9937fee9-a8bb-43da-91c5-2cb8e7e7dab8 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:27.757657 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:27.815256 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 54/107] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:53:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 18 16:53:27.820790 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b83bbd-f101-437a-a257-eaae5ed74888 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:27.951467 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed7b30b-a6c3-4650-a460-337612426266 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:27.951837 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed7b30b-a6c3-4650-a460-337612426266 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:27.952123 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed7b30b-a6c3-4650-a460-337612426266 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:27.952348 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b83bbd-f101-437a-a257-eaae5ed74888 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:27.953544 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 55/108] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:53:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:53:27.959978 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8373d0c-3379-4f71-a47e-a7708055f700 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:27.960249 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8373d0c-3379-4f71-a47e-a7708055f700 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:27.960527 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8373d0c-3379-4f71-a47e-a7708055f700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:27.988760 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-e8373d0c-3379-4f71-a47e-a7708055f700 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:53:27.989976 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-e8373d0c-3379-4f71-a47e-a7708055f700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:53:27.991854 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575ca288-5cb4-4db8-b40d-2ff9e9c3c459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:27.992616 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575ca288-5cb4-4db8-b40d-2ff9e9c3c459 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:27.992996 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575ca288-5cb4-4db8-b40d-2ff9e9c3c459 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:27.993284 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575ca288-5cb4-4db8-b40d-2ff9e9c3c459 None admin] PATH_INFO: `/v3/projects/62beed06c75a4725ae7834995533d0a0` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:27.994430 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575ca288-5cb4-4db8-b40d-2ff9e9c3c459 None admin] RBAC: Authorizing `identity:get_project(project_id=62beed06c75a4725ae7834995533d0a0)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:27.995936 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575ca288-5cb4-4db8-b40d-2ff9e9c3c459 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:27.997568 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 54/109] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 16:53:27 2022] GET /identity/v3/projects/62beed06c75a4725ae7834995533d0a0 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:27.998051 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:28.004132 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575ca288-5cb4-4db8-b40d-2ff9e9c3c459 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:28.106815 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 56/110] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:53:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:53:28.132623 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575ca288-5cb4-4db8-b40d-2ff9e9c3c459 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:28.173042 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33ed3ff-d8e7-4ab8-abcd-2c5e03844d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:28.173669 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ed3ff-d8e7-4ab8-abcd-2c5e03844d9c None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:28.175083 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ed3ff-d8e7-4ab8-abcd-2c5e03844d9c None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:28.175550 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ed3ff-d8e7-4ab8-abcd-2c5e03844d9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:28.176292 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33ed3ff-d8e7-4ab8-abcd-2c5e03844d9c None admin] RBAC: Adding query filter params (role.id=d49353712e134598b13b44d21b1e791e, scope.project.id=62beed06c75a4725ae7834995533d0a0, user.id=f601719176bc414ab763d3f1fd6662d8) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:53:28.176605 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33ed3ff-d8e7-4ab8-abcd-2c5e03844d9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:28.178067 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33ed3ff-d8e7-4ab8-abcd-2c5e03844d9c None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:28.184116 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 55/111] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 16:53:28 2022] GET /identity/v3/role_assignments?user.id=f601719176bc414ab763d3f1fd6662d8&scope.project.id=62beed06c75a4725ae7834995533d0a0&role.id=d49353712e134598b13b44d21b1e791e => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:53:28.431559 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e5ecd-cc0b-4fbf-9f3c-d1df78647afa None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:28.432057 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e5ecd-cc0b-4fbf-9f3c-d1df78647afa None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:28.432157 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e5ecd-cc0b-4fbf-9f3c-d1df78647afa None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:28.433734 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 57/112] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:53:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:53:28.448249 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f876e4ce-b3fe-4dc5-89b3-8da5acf164db None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:28.448674 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f876e4ce-b3fe-4dc5-89b3-8da5acf164db None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:28.448881 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f876e4ce-b3fe-4dc5-89b3-8da5acf164db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:28.471513 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f466e0e5-e626-4f85-8cdd-a0ca88299f5a None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:28.471882 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f466e0e5-e626-4f85-8cdd-a0ca88299f5a None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:28.472059 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f466e0e5-e626-4f85-8cdd-a0ca88299f5a None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:28.473514 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 58/113] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:53:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 16:53:28.474228 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-f876e4ce-b3fe-4dc5-89b3-8da5acf164db None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:53:28.475557 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-f876e4ce-b3fe-4dc5-89b3-8da5acf164db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:53:28.479026 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f3d2c4-4fe6-490f-9599-614e27dad83f None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:28.479312 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f3d2c4-4fe6-490f-9599-614e27dad83f None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:28.479587 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f3d2c4-4fe6-490f-9599-614e27dad83f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:28.508595 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-d7f3d2c4-4fe6-490f-9599-614e27dad83f None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:53:28.510121 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-d7f3d2c4-4fe6-490f-9599-614e27dad83f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:53:28.560359 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:28.575450 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 56/114] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:53:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:53:28.577151 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837777f4-2192-4761-8ee4-faeb29149db9 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:28.577437 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837777f4-2192-4761-8ee4-faeb29149db9 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:28.577713 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837777f4-2192-4761-8ee4-faeb29149db9 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:28.579648 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 57/115] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:53:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:53:28.581466 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837777f4-2192-4761-8ee4-faeb29149db9 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:28.585403 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 59/116] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:53:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 16:53:28.591576 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1ff80-280c-4bab-bc01-383217b7c330 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:28.592179 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1ff80-280c-4bab-bc01-383217b7c330 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:28.592651 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1ff80-280c-4bab-bc01-383217b7c330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:28.629342 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-20a1ff80-280c-4bab-bc01-383217b7c330 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:53:28.632144 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-20a1ff80-280c-4bab-bc01-383217b7c330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:53:28.663112 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:28.744493 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 60/117] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:53:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:53:28.787664 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837777f4-2192-4761-8ee4-faeb29149db9 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:28.841292 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f598634-7f93-4dd6-ba59-87657a1a2c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:28.842334 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f598634-7f93-4dd6-ba59-87657a1a2c03 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:28.842859 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f598634-7f93-4dd6-ba59-87657a1a2c03 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:28.843340 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f598634-7f93-4dd6-ba59-87657a1a2c03 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:28.875782 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f598634-7f93-4dd6-ba59-87657a1a2c03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:53:28.880624 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-1f598634-7f93-4dd6-ba59-87657a1a2c03 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 16:53:28.880624 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:53:28.880624 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:53:28.880624 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:53:28.880624 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:53:28.880624 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:53:28.880624 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:53:28.880624 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:53:28.880624 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:53:28.880624 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:53:28.880624 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:53:28.880624 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:53:28.880624 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:53:28.880624 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:53:28.880624 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:53:28.880624 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:53:28.881497 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:53:28.881497 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:53:28.881497 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:53:28.881497 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:53:28.881497 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:53:28.881497 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:53:28.881497 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:53:28.881497 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:53:28.881497 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:53:28.881497 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:53:28.881497 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:53:28.881497 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:53:28.881497 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:53:28.881497 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:53:28.881497 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:53:28.881497 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:53:28.882228 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:53:28.882228 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:53:28.882228 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:53:28.882228 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:53:28.882228 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:53:28.882228 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:53:28.882228 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:53:28.882228 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:53:28.882228 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:53:28.882228 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:53:28.882228 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 16:53:28.882228 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:53:28.882228 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 58/118] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:53:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:53:28.889429 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a1ff80-280c-4bab-bc01-383217b7c330 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:28.996095 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a1ff80-280c-4bab-bc01-383217b7c330 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:29.033440 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5fef5a-8511-4ddf-b71f-f5b867b0bb53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:29.034507 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5fef5a-8511-4ddf-b71f-f5b867b0bb53 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:29.035053 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5fef5a-8511-4ddf-b71f-f5b867b0bb53 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:29.035521 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5fef5a-8511-4ddf-b71f-f5b867b0bb53 None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:29.035945 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f598634-7f93-4dd6-ba59-87657a1a2c03 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:29.036681 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5fef5a-8511-4ddf-b71f-f5b867b0bb53 None admin] RBAC: Adding query filter params (name=member) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:53:29.037177 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5fef5a-8511-4ddf-b71f-f5b867b0bb53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:29.038755 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5fef5a-8511-4ddf-b71f-f5b867b0bb53 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:29.048372 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 61/119] 10.222.0.82 () {56 vars in 1101 bytes} [Thu Aug 18 16:53:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:29.056570 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5fef5a-8511-4ddf-b71f-f5b867b0bb53 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:29.071525 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:29.137096 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:29.163008 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5fef5a-8511-4ddf-b71f-f5b867b0bb53 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:29.189227 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f598634-7f93-4dd6-ba59-87657a1a2c03 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:29.199795 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a33f0c1-08b5-42a8-925e-9de499f35b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:29.200738 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a33f0c1-08b5-42a8-925e-9de499f35b2e None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:29.201213 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a33f0c1-08b5-42a8-925e-9de499f35b2e None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:29.201678 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a33f0c1-08b5-42a8-925e-9de499f35b2e None admin] PATH_INFO: `/v3/users/3f70ad03afa246c393b5448fe7ee6860` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:29.203070 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a33f0c1-08b5-42a8-925e-9de499f35b2e None admin] RBAC: Authorizing `identity:get_user(user_id=3f70ad03afa246c393b5448fe7ee6860)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:29.204708 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a33f0c1-08b5-42a8-925e-9de499f35b2e None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:29.206990 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 62/120] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 16:53:29 2022] GET /identity/v3/users/3f70ad03afa246c393b5448fe7ee6860 => generated 275 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:29.207975 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a33f0c1-08b5-42a8-925e-9de499f35b2e None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:29.228147 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f927880a-9d71-423b-a541-d17475797c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:29.228782 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927880a-9d71-423b-a541-d17475797c75 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:29.229064 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927880a-9d71-423b-a541-d17475797c75 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:29.229336 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927880a-9d71-423b-a541-d17475797c75 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:29.240985 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f927880a-9d71-423b-a541-d17475797c75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:53:29.245725 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-f927880a-9d71-423b-a541-d17475797c75 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 16:53:29.245725 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:53:29.245725 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:53:29.245725 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:53:29.245725 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:53:29.245725 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:53:29.245725 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:53:29.245725 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:53:29.245725 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:53:29.245725 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:53:29.245725 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:53:29.245725 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:53:29.245725 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:53:29.245725 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:53:29.245725 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:53:29.245725 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:53:29.246585 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:53:29.246585 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:53:29.246585 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:53:29.246585 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:53:29.246585 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:53:29.246585 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:53:29.246585 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:53:29.246585 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:53:29.246585 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:53:29.246585 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:53:29.246585 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:53:29.246585 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:53:29.246585 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:53:29.246585 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:53:29.246585 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:53:29.246585 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:53:29.247356 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:53:29.247356 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:53:29.247356 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:53:29.247356 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:53:29.247356 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:53:29.247356 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:53:29.247356 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:53:29.247356 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:53:29.247356 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:53:29.247356 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:53:29.247356 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 16:53:29.247356 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:53:29.247356 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 59/121] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 16:53:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:53:29.247989 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f927880a-9d71-423b-a541-d17475797c75 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:29.305453 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:29.328818 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a33f0c1-08b5-42a8-925e-9de499f35b2e None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:29.366614 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fa67fb-b198-4358-8fcd-c0febd980002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:29.367724 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa67fb-b198-4358-8fcd-c0febd980002 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:29.368225 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa67fb-b198-4358-8fcd-c0febd980002 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:29.368691 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa67fb-b198-4358-8fcd-c0febd980002 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:29.380040 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fa67fb-b198-4358-8fcd-c0febd980002 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:53:29.382053 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f927880a-9d71-423b-a541-d17475797c75 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:29.385193 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-88fa67fb-b198-4358-8fcd-c0febd980002 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 16:53:29.385193 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:53:29.385193 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:53:29.385193 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:53:29.385193 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:53:29.385193 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:53:29.385193 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:53:29.385193 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:53:29.385193 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:53:29.385193 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:53:29.385193 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:53:29.385193 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:53:29.385193 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:53:29.385193 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:53:29.385193 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:53:29.385193 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:53:29.386111 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:53:29.386111 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:53:29.386111 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:53:29.386111 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:53:29.386111 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:53:29.386111 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:53:29.386111 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:53:29.386111 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:53:29.386111 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:53:29.386111 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:53:29.386111 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:53:29.386111 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:53:29.386111 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:53:29.386111 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:53:29.386111 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:53:29.386111 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:53:29.386843 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:53:29.386843 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:53:29.386843 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:53:29.386843 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:53:29.386843 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:53:29.386843 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:53:29.386843 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:53:29.386843 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:53:29.386843 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:53:29.386843 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:53:29.386843 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 16:53:29.386843 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:53:29.389306 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 63/122] 10.222.0.82 () {56 vars in 1114 bytes} [Thu Aug 18 16:53:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:53:29.390442 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fa67fb-b198-4358-8fcd-c0febd980002 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:29.405301 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:29.423182 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329dfcb1-f84d-4dab-83f4-2f462a68a276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:29.424345 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329dfcb1-f84d-4dab-83f4-2f462a68a276 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:29.424812 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329dfcb1-f84d-4dab-83f4-2f462a68a276 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:29.425224 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329dfcb1-f84d-4dab-83f4-2f462a68a276 None admin] PATH_INFO: `/v3/projects/62beed06c75a4725ae7834995533d0a0` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:29.426359 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329dfcb1-f84d-4dab-83f4-2f462a68a276 None admin] RBAC: Authorizing `identity:get_project(project_id=62beed06c75a4725ae7834995533d0a0)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:29.427872 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329dfcb1-f84d-4dab-83f4-2f462a68a276 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:29.429647 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 60/123] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 16:53:29 2022] GET /identity/v3/projects/62beed06c75a4725ae7834995533d0a0 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:29.434909 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329dfcb1-f84d-4dab-83f4-2f462a68a276 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:29.498685 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fa67fb-b198-4358-8fcd-c0febd980002 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:29.536266 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe8aaa7-aa39-4008-8f3a-53e6251e58a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:29.537177 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe8aaa7-aa39-4008-8f3a-53e6251e58a5 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:29.537682 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe8aaa7-aa39-4008-8f3a-53e6251e58a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:29.538148 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe8aaa7-aa39-4008-8f3a-53e6251e58a5 None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:29.539094 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe8aaa7-aa39-4008-8f3a-53e6251e58a5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:53:29.539699 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe8aaa7-aa39-4008-8f3a-53e6251e58a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:29.540735 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329dfcb1-f84d-4dab-83f4-2f462a68a276 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:29.542061 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe8aaa7-aa39-4008-8f3a-53e6251e58a5 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:29.551644 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 64/124] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:53:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:29.552726 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe8aaa7-aa39-4008-8f3a-53e6251e58a5 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:29.555519 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:29.582859 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc17c1b0-0961-4a06-a237-0ec53e5533c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:29.583530 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc17c1b0-0961-4a06-a237-0ec53e5533c0 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:29.583919 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc17c1b0-0961-4a06-a237-0ec53e5533c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:29.584204 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc17c1b0-0961-4a06-a237-0ec53e5533c0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:29.595502 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc17c1b0-0961-4a06-a237-0ec53e5533c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:53:29.601923 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-bc17c1b0-0961-4a06-a237-0ec53e5533c0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 16:53:29.601923 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:53:29.601923 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:53:29.601923 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:53:29.601923 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:53:29.601923 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:53:29.601923 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:53:29.601923 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:53:29.601923 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:53:29.601923 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:53:29.601923 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:53:29.601923 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:53:29.601923 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:53:29.601923 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:53:29.601923 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:53:29.601923 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:53:29.602766 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:53:29.602766 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:53:29.602766 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:53:29.602766 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:53:29.602766 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:53:29.602766 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:53:29.602766 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:53:29.602766 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:53:29.602766 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:53:29.602766 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:53:29.602766 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:53:29.602766 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:53:29.602766 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:53:29.602766 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:53:29.602766 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:53:29.602766 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:53:29.604091 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:53:29.604091 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:53:29.604091 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:53:29.604091 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:53:29.604091 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:53:29.604091 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:53:29.604091 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:53:29.604091 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:53:29.604091 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:53:29.604091 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:53:29.604091 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 16:53:29.604091 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:53:29.604091 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 61/125] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:53:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:53:29.604091 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc17c1b0-0961-4a06-a237-0ec53e5533c0 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:29.651389 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:29.667045 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe8aaa7-aa39-4008-8f3a-53e6251e58a5 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:29.703940 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a87f1b5-292a-487f-872e-5a4d881671fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:29.704890 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87f1b5-292a-487f-872e-5a4d881671fa None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:29.705393 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87f1b5-292a-487f-872e-5a4d881671fa None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:29.705887 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87f1b5-292a-487f-872e-5a4d881671fa None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:29.706753 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a87f1b5-292a-487f-872e-5a4d881671fa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:53:29.707273 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a87f1b5-292a-487f-872e-5a4d881671fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:29.708860 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a87f1b5-292a-487f-872e-5a4d881671fa None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:29.713876 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc17c1b0-0961-4a06-a237-0ec53e5533c0 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:29.717817 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 65/126] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 16:53:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:29.719106 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a87f1b5-292a-487f-872e-5a4d881671fa None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:29.756235 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f38a4a5-d66f-4654-b2b0-a60a59ffdbc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:29.756824 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f38a4a5-d66f-4654-b2b0-a60a59ffdbc1 None admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:29.757107 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f38a4a5-d66f-4654-b2b0-a60a59ffdbc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:29.757381 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f38a4a5-d66f-4654-b2b0-a60a59ffdbc1 None admin] PATH_INFO: `/v3/projects/62beed06c75a4725ae7834995533d0a0/users/3f70ad03afa246c393b5448fe7ee6860/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:29.759628 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f38a4a5-d66f-4654-b2b0-a60a59ffdbc1 None admin] RBAC: Authorizing `identity:create_grant(project_id=62beed06c75a4725ae7834995533d0a0, user_id=3f70ad03afa246c393b5448fe7ee6860, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:29.760896 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f38a4a5-d66f-4654-b2b0-a60a59ffdbc1 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:29.791133 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:29.828076 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a87f1b5-292a-487f-872e-5a4d881671fa None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:29.841602 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 62/127] 10.222.0.82 () {58 vars in 1439 bytes} [Thu Aug 18 16:53:29 2022] PUT /identity/v3/projects/62beed06c75a4725ae7834995533d0a0/users/3f70ad03afa246c393b5448fe7ee6860/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 16:53:29.842877 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f38a4a5-d66f-4654-b2b0-a60a59ffdbc1 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:29.886522 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032b7a54-f6b9-4208-9288-86ca705deece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:29.887857 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032b7a54-f6b9-4208-9288-86ca705deece None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:29.889624 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032b7a54-f6b9-4208-9288-86ca705deece None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:29.890142 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032b7a54-f6b9-4208-9288-86ca705deece None admin] PATH_INFO: `/v3/users/329c213a845c4ab2a0642e5a7dbd4b18` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:29.892363 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032b7a54-f6b9-4208-9288-86ca705deece None admin] RBAC: Authorizing `identity:get_user(user_id=329c213a845c4ab2a0642e5a7dbd4b18)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:29.896300 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032b7a54-f6b9-4208-9288-86ca705deece None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:29.899812 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 66/128] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 16:53:29 2022] GET /identity/v3/users/329c213a845c4ab2a0642e5a7dbd4b18 => generated 247 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:29.901979 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032b7a54-f6b9-4208-9288-86ca705deece None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:30.051472 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f38a4a5-d66f-4654-b2b0-a60a59ffdbc1 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:30.055292 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:30.083683 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032b7a54-f6b9-4208-9288-86ca705deece None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:30.098995 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77932a50-d9bf-41d9-80a1-4ed1807e769b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:30.099964 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77932a50-d9bf-41d9-80a1-4ed1807e769b None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:30.100558 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77932a50-d9bf-41d9-80a1-4ed1807e769b None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:30.100984 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77932a50-d9bf-41d9-80a1-4ed1807e769b None admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:30.102317 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77932a50-d9bf-41d9-80a1-4ed1807e769b None admin] RBAC: Adding query filter params (name=member) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:53:30.102947 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77932a50-d9bf-41d9-80a1-4ed1807e769b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:30.105063 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77932a50-d9bf-41d9-80a1-4ed1807e769b None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:30.117915 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 63/129] 10.222.0.82 () {56 vars in 1101 bytes} [Thu Aug 18 16:53:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:30.126947 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77932a50-d9bf-41d9-80a1-4ed1807e769b None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:30.131286 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:30.143663 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d17ad35-44a8-4c0c-8d4d-27c4f614a8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:30.144388 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d17ad35-44a8-4c0c-8d4d-27c4f614a8fa None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:30.144672 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d17ad35-44a8-4c0c-8d4d-27c4f614a8fa None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:30.144943 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d17ad35-44a8-4c0c-8d4d-27c4f614a8fa None admin] PATH_INFO: `/v3/users/3f70ad03afa246c393b5448fe7ee6860` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:30.148038 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d17ad35-44a8-4c0c-8d4d-27c4f614a8fa None admin] RBAC: Authorizing `identity:get_user(user_id=3f70ad03afa246c393b5448fe7ee6860)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:30.149609 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d17ad35-44a8-4c0c-8d4d-27c4f614a8fa None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:30.151546 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 67/130] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 16:53:29 2022] GET /identity/v3/users/3f70ad03afa246c393b5448fe7ee6860 => generated 275 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:30.159158 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d17ad35-44a8-4c0c-8d4d-27c4f614a8fa None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:30.291875 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d17ad35-44a8-4c0c-8d4d-27c4f614a8fa None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:30.295366 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77932a50-d9bf-41d9-80a1-4ed1807e769b None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:30.337261 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2088fb-84ff-4f4d-8995-e9a5509d359f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:30.337931 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2088fb-84ff-4f4d-8995-e9a5509d359f None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:30.338224 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2088fb-84ff-4f4d-8995-e9a5509d359f None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:30.338501 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2088fb-84ff-4f4d-8995-e9a5509d359f None admin] PATH_INFO: `/v3/users/3f70ad03afa246c393b5448fe7ee6860` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:30.339838 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2088fb-84ff-4f4d-8995-e9a5509d359f None admin] RBAC: Authorizing `identity:get_user(user_id=3f70ad03afa246c393b5448fe7ee6860)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:30.341541 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2088fb-84ff-4f4d-8995-e9a5509d359f None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:30.342021 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abb521b-d9b4-4686-91a3-f2a9eb1b42c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:30.342918 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abb521b-d9b4-4686-91a3-f2a9eb1b42c3 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:30.343282 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 68/131] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 16:53:30 2022] GET /identity/v3/users/3f70ad03afa246c393b5448fe7ee6860 => generated 275 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:30.343655 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abb521b-d9b4-4686-91a3-f2a9eb1b42c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:30.344289 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2088fb-84ff-4f4d-8995-e9a5509d359f None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:30.344641 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abb521b-d9b4-4686-91a3-f2a9eb1b42c3 None admin] PATH_INFO: `/v3/projects/62beed06c75a4725ae7834995533d0a0` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:30.346097 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abb521b-d9b4-4686-91a3-f2a9eb1b42c3 None admin] RBAC: Authorizing `identity:get_project(project_id=62beed06c75a4725ae7834995533d0a0)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:30.349000 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abb521b-d9b4-4686-91a3-f2a9eb1b42c3 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:30.351039 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 64/132] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 16:53:30 2022] GET /identity/v3/projects/62beed06c75a4725ae7834995533d0a0 => generated 298 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:30.352564 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032eafcd-4fcc-4863-ba37-9cb04c1771a3 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:30.353060 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032eafcd-4fcc-4863-ba37-9cb04c1771a3 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:30.353594 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032eafcd-4fcc-4863-ba37-9cb04c1771a3 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:30.355637 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 65/133] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:53:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:53:30.360807 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032eafcd-4fcc-4863-ba37-9cb04c1771a3 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:30.393484 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:30.425186 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:30.460208 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2088fb-84ff-4f4d-8995-e9a5509d359f None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:30.467867 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032eafcd-4fcc-4863-ba37-9cb04c1771a3 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:30.497196 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9930bb05-7e47-481a-b974-43b1e2f081d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:30.498203 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9930bb05-7e47-481a-b974-43b1e2f081d7 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:30.498728 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9930bb05-7e47-481a-b974-43b1e2f081d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:30.499649 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9930bb05-7e47-481a-b974-43b1e2f081d7 None admin] PATH_INFO: `/v3/projects/62beed06c75a4725ae7834995533d0a0` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:30.501129 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9930bb05-7e47-481a-b974-43b1e2f081d7 None admin] RBAC: Authorizing `identity:get_project(project_id=62beed06c75a4725ae7834995533d0a0)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:30.503435 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9930bb05-7e47-481a-b974-43b1e2f081d7 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:30.504083 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9852b094-f674-4674-af5c-2068c344e79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:30.504733 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9852b094-f674-4674-af5c-2068c344e79a None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:30.505007 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9852b094-f674-4674-af5c-2068c344e79a None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:30.505289 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9852b094-f674-4674-af5c-2068c344e79a None admin] PATH_INFO: `/v3/projects/244b82f9ef034b6bb3d250becacf818d` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:30.506311 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 69/134] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 16:53:30 2022] GET /identity/v3/projects/62beed06c75a4725ae7834995533d0a0 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:30.506714 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9852b094-f674-4674-af5c-2068c344e79a None admin] RBAC: Authorizing `identity:get_project(project_id=244b82f9ef034b6bb3d250becacf818d)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:30.507585 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9930bb05-7e47-481a-b974-43b1e2f081d7 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:30.508220 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9852b094-f674-4674-af5c-2068c344e79a None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:30.509845 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 66/135] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 16:53:30 2022] GET /identity/v3/projects/244b82f9ef034b6bb3d250becacf818d => generated 312 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:30.515036 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa0f17-53b2-42ac-aa06-17aac20c1c9d None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:30.515366 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa0f17-53b2-42ac-aa06-17aac20c1c9d None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:30.515672 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa0f17-53b2-42ac-aa06-17aac20c1c9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:30.534375 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-64aa0f17-53b2-42ac-aa06-17aac20c1c9d None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:53:30.535605 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-64aa0f17-53b2-42ac-aa06-17aac20c1c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:53:30.614851 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 67/136] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:53:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:53:30.615644 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aa0f17-53b2-42ac-aa06-17aac20c1c9d None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:30.621042 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9930bb05-7e47-481a-b974-43b1e2f081d7 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:30.657291 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc52a33-917a-421a-9c93-e0f239b98362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:30.658710 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc52a33-917a-421a-9c93-e0f239b98362 None admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:30.659176 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc52a33-917a-421a-9c93-e0f239b98362 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:30.659522 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc52a33-917a-421a-9c93-e0f239b98362 None admin] PATH_INFO: `/v3/projects/62beed06c75a4725ae7834995533d0a0/users/329c213a845c4ab2a0642e5a7dbd4b18/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:30.659943 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:30.663406 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc52a33-917a-421a-9c93-e0f239b98362 None admin] RBAC: Authorizing `identity:create_grant(project_id=62beed06c75a4725ae7834995533d0a0, user_id=329c213a845c4ab2a0642e5a7dbd4b18, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:30.665367 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc52a33-917a-421a-9c93-e0f239b98362 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:30.670111 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:30.737704 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aa0f17-53b2-42ac-aa06-17aac20c1c9d None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:30.778024 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 70/137] 10.222.0.82 () {58 vars in 1439 bytes} [Thu Aug 18 16:53:30 2022] PUT /identity/v3/projects/62beed06c75a4725ae7834995533d0a0/users/329c213a845c4ab2a0642e5a7dbd4b18/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 16:53:30.778894 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc43fb7-f0e1-4b86-b6ee-db38433a8a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:30.781867 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc43fb7-f0e1-4b86-b6ee-db38433a8a53 None admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:30.781867 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc43fb7-f0e1-4b86-b6ee-db38433a8a53 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:30.781867 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc43fb7-f0e1-4b86-b6ee-db38433a8a53 None admin] PATH_INFO: `/v3/projects/62beed06c75a4725ae7834995533d0a0/users/3f70ad03afa246c393b5448fe7ee6860/roles/f41f6f29d9de4b1985c8c8c14f002cf2` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:30.782850 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc43fb7-f0e1-4b86-b6ee-db38433a8a53 None admin] RBAC: Authorizing `identity:create_grant(project_id=62beed06c75a4725ae7834995533d0a0, user_id=3f70ad03afa246c393b5448fe7ee6860, role_id=f41f6f29d9de4b1985c8c8c14f002cf2)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:30.785073 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc43fb7-f0e1-4b86-b6ee-db38433a8a53 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:30.785957 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc52a33-917a-421a-9c93-e0f239b98362 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:30.898426 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 68/138] 10.222.0.82 () {58 vars in 1439 bytes} [Thu Aug 18 16:53:30 2022] PUT /identity/v3/projects/62beed06c75a4725ae7834995533d0a0/users/3f70ad03afa246c393b5448fe7ee6860/roles/f41f6f29d9de4b1985c8c8c14f002cf2 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 16:53:30.900884 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e57e95-5f00-4832-aebc-dfd6b8ceeb9e None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:30.901467 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e57e95-5f00-4832-aebc-dfd6b8ceeb9e None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:30.901823 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e57e95-5f00-4832-aebc-dfd6b8ceeb9e None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:30.909636 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 69/139] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:53:30 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:53:30.912678 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc52a33-917a-421a-9c93-e0f239b98362 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:30.916421 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb682bf-486f-47ca-b37c-468d6e09098c None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:30.916421 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb682bf-486f-47ca-b37c-468d6e09098c None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:30.916741 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb682bf-486f-47ca-b37c-468d6e09098c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:30.949532 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-3fb682bf-486f-47ca-b37c-468d6e09098c None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:53:30.951005 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-3fb682bf-486f-47ca-b37c-468d6e09098c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:53:30.957337 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df28aeaf-e0c0-4272-a8ec-e7e35ad28aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:30.959347 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df28aeaf-e0c0-4272-a8ec-e7e35ad28aae None admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:30.960019 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df28aeaf-e0c0-4272-a8ec-e7e35ad28aae None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:30.960436 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df28aeaf-e0c0-4272-a8ec-e7e35ad28aae None admin] PATH_INFO: `/v3/projects/244b82f9ef034b6bb3d250becacf818d/users/3f70ad03afa246c393b5448fe7ee6860/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:30.965329 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df28aeaf-e0c0-4272-a8ec-e7e35ad28aae None admin] RBAC: Authorizing `identity:create_grant(project_id=244b82f9ef034b6bb3d250becacf818d, user_id=3f70ad03afa246c393b5448fe7ee6860, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:30.969809 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df28aeaf-e0c0-4272-a8ec-e7e35ad28aae None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:31.046373 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 70/140] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:53:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:53:31.047369 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb682bf-486f-47ca-b37c-468d6e09098c None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:31.057023 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 71/141] 10.222.0.82 () {58 vars in 1439 bytes} [Thu Aug 18 16:53:30 2022] PUT /identity/v3/projects/244b82f9ef034b6bb3d250becacf818d/users/3f70ad03afa246c393b5448fe7ee6860/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 16:53:31.072304 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:31.167541 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb682bf-486f-47ca-b37c-468d6e09098c None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:31.205862 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096061d6-f9c6-4c2a-9680-4ed0337642f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:31.206772 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096061d6-f9c6-4c2a-9680-4ed0337642f9 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:31.207279 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096061d6-f9c6-4c2a-9680-4ed0337642f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:31.207783 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096061d6-f9c6-4c2a-9680-4ed0337642f9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:31.209097 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096061d6-f9c6-4c2a-9680-4ed0337642f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:31.210774 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096061d6-f9c6-4c2a-9680-4ed0337642f9 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:31.212961 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 71/142] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:53:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:31.218566 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df28aeaf-e0c0-4272-a8ec-e7e35ad28aae None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:31.282777 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:31.330867 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df28aeaf-e0c0-4272-a8ec-e7e35ad28aae None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:31.366774 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9678c729-b2ea-4214-ac69-8e5a4ecc705f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:31.367434 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9678c729-b2ea-4214-ac69-8e5a4ecc705f None admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:31.367719 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9678c729-b2ea-4214-ac69-8e5a4ecc705f None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:31.367998 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9678c729-b2ea-4214-ac69-8e5a4ecc705f None admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:31.368782 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9678c729-b2ea-4214-ac69-8e5a4ecc705f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:31.370248 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9678c729-b2ea-4214-ac69-8e5a4ecc705f None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:31.400212 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096061d6-f9c6-4c2a-9680-4ed0337642f9 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:31.488984 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 72/143] 10.222.0.82 () {60 vars in 1132 bytes} [Thu Aug 18 16:53:31 2022] POST /identity/v3/users => generated 283 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 16:53:31.505478 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:31.518827 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096061d6-f9c6-4c2a-9680-4ed0337642f9 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:31.557355 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48b5a31-ee63-4e7f-b155-cbe02ecbec0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:31.558019 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b5a31-ee63-4e7f-b155-cbe02ecbec0d None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:31.558317 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b5a31-ee63-4e7f-b155-cbe02ecbec0d None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:31.558578 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b5a31-ee63-4e7f-b155-cbe02ecbec0d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:31.570908 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48b5a31-ee63-4e7f-b155-cbe02ecbec0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:53:31.575935 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-d48b5a31-ee63-4e7f-b155-cbe02ecbec0d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 16:53:31.575935 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:53:31.575935 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:53:31.575935 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:53:31.575935 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:53:31.575935 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:53:31.575935 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:53:31.575935 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:53:31.575935 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:53:31.575935 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:53:31.575935 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:53:31.575935 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:53:31.575935 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:53:31.575935 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:53:31.575935 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:53:31.575935 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:53:31.576802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:53:31.576802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:53:31.576802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:53:31.576802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:53:31.576802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:53:31.576802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:53:31.576802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:53:31.576802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:53:31.576802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:53:31.576802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:53:31.576802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:53:31.576802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:53:31.576802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:53:31.576802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:53:31.576802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:53:31.576802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:53:31.577523 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:53:31.577523 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:53:31.577523 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:53:31.577523 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:53:31.577523 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:53:31.577523 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:53:31.577523 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:53:31.577523 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:53:31.577523 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:53:31.577523 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:53:31.577523 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 16:53:31.577523 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:53:31.577523 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 72/144] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:53:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 16:53:31.583430 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9678c729-b2ea-4214-ac69-8e5a4ecc705f None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:31.712671 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9678c729-b2ea-4214-ac69-8e5a4ecc705f None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:31.733531 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:31.759356 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f893bf3a-2a11-4f32-926a-9b836b72eab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:31.759356 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f893bf3a-2a11-4f32-926a-9b836b72eab2 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:31.759356 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f893bf3a-2a11-4f32-926a-9b836b72eab2 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:31.759356 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f893bf3a-2a11-4f32-926a-9b836b72eab2 None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:31.760064 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f893bf3a-2a11-4f32-926a-9b836b72eab2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:53:31.760223 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f893bf3a-2a11-4f32-926a-9b836b72eab2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:31.761557 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f893bf3a-2a11-4f32-926a-9b836b72eab2 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:31.771064 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 73/145] 10.222.0.82 () {56 vars in 1101 bytes} [Thu Aug 18 16:53:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:31.777982 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48b5a31-ee63-4e7f-b155-cbe02ecbec0d None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:31.919239 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48b5a31-ee63-4e7f-b155-cbe02ecbec0d None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:31.960645 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604749c1-595b-47f4-9abd-8e34a1c9fe75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:31.961294 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604749c1-595b-47f4-9abd-8e34a1c9fe75 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:31.961581 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604749c1-595b-47f4-9abd-8e34a1c9fe75 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:31.961861 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604749c1-595b-47f4-9abd-8e34a1c9fe75 None admin] PATH_INFO: `/v3/users/f601719176bc414ab763d3f1fd6662d8` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:31.963091 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604749c1-595b-47f4-9abd-8e34a1c9fe75 None admin] RBAC: Authorizing `identity:get_user(user_id=f601719176bc414ab763d3f1fd6662d8)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:31.964572 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604749c1-595b-47f4-9abd-8e34a1c9fe75 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:31.966286 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 73/146] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 16:53:31 2022] GET /identity/v3/users/f601719176bc414ab763d3f1fd6662d8 => generated 289 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:31.972230 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f893bf3a-2a11-4f32-926a-9b836b72eab2 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:32.089799 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f893bf3a-2a11-4f32-926a-9b836b72eab2 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:32.132959 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6759a74-4ceb-4375-b787-0d02d43e090c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:32.133613 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6759a74-4ceb-4375-b787-0d02d43e090c None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:32.133898 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6759a74-4ceb-4375-b787-0d02d43e090c None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:32.134174 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6759a74-4ceb-4375-b787-0d02d43e090c None admin] PATH_INFO: `/v3/projects/62beed06c75a4725ae7834995533d0a0` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:32.135410 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6759a74-4ceb-4375-b787-0d02d43e090c None admin] RBAC: Authorizing `identity:get_project(project_id=62beed06c75a4725ae7834995533d0a0)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:32.136928 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6759a74-4ceb-4375-b787-0d02d43e090c None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:32.138880 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 74/147] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 16:53:31 2022] GET /identity/v3/projects/62beed06c75a4725ae7834995533d0a0 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:32.145360 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604749c1-595b-47f4-9abd-8e34a1c9fe75 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:32.163742 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:32.258694 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604749c1-595b-47f4-9abd-8e34a1c9fe75 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:32.299484 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361b3805-0e90-4702-a3d3-b3a928559bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:32.300116 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b3805-0e90-4702-a3d3-b3a928559bd9 None admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:32.300403 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b3805-0e90-4702-a3d3-b3a928559bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:32.300686 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b3805-0e90-4702-a3d3-b3a928559bd9 None admin] PATH_INFO: `/v3/projects/62beed06c75a4725ae7834995533d0a0/users/f601719176bc414ab763d3f1fd6662d8/roles/d49353712e134598b13b44d21b1e791e` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:32.302728 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361b3805-0e90-4702-a3d3-b3a928559bd9 None admin] RBAC: Authorizing `identity:create_grant(project_id=62beed06c75a4725ae7834995533d0a0, user_id=f601719176bc414ab763d3f1fd6662d8, role_id=d49353712e134598b13b44d21b1e791e)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:32.304295 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361b3805-0e90-4702-a3d3-b3a928559bd9 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:32.338565 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec4110-53d3-4c0f-ba3a-4083ceff346d None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:32.338941 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec4110-53d3-4c0f-ba3a-4083ceff346d None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:32.339235 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec4110-53d3-4c0f-ba3a-4083ceff346d None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:32.340806 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 75/148] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:53:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:53:32.349702 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de3b808-2b04-47de-b5be-cfc2e9d195f7 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:32.349993 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de3b808-2b04-47de-b5be-cfc2e9d195f7 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:32.350277 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de3b808-2b04-47de-b5be-cfc2e9d195f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:32.370670 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-3de3b808-2b04-47de-b5be-cfc2e9d195f7 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:53:32.371934 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-3de3b808-2b04-47de-b5be-cfc2e9d195f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:53:32.385400 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 74/149] 10.222.0.82 () {58 vars in 1439 bytes} [Thu Aug 18 16:53:32 2022] PUT /identity/v3/projects/62beed06c75a4725ae7834995533d0a0/users/f601719176bc414ab763d3f1fd6662d8/roles/d49353712e134598b13b44d21b1e791e => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 16:53:32.426480 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:32.472528 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 76/150] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:53:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:53:33.073560 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361b3805-0e90-4702-a3d3-b3a928559bd9 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:33.165136 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:33.192819 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361b3805-0e90-4702-a3d3-b3a928559bd9 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:33.229726 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20510ea5-d4be-4165-9581-7d68299dd1ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:33.230722 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20510ea5-d4be-4165-9581-7d68299dd1ca None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:33.231217 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20510ea5-d4be-4165-9581-7d68299dd1ca None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:33.231703 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20510ea5-d4be-4165-9581-7d68299dd1ca None admin] PATH_INFO: `/v3/roles/member` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:33.243344 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20510ea5-d4be-4165-9581-7d68299dd1ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:53:33.249001 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-20510ea5-d4be-4165-9581-7d68299dd1ca None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 16:53:33.249001 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:53:33.249001 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:53:33.249001 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:53:33.249001 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:53:33.249001 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:53:33.249001 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:53:33.249001 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:53:33.249001 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:53:33.249001 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:53:33.249001 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:53:33.249001 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:53:33.249001 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:53:33.249001 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:53:33.249001 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:53:33.249001 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:53:33.249802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:53:33.249802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:53:33.249802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:53:33.249802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:53:33.249802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:53:33.249802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:53:33.249802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:53:33.249802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:53:33.249802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:53:33.249802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:53:33.249802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:53:33.249802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:53:33.249802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:53:33.249802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:53:33.249802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:53:33.249802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:53:33.250535 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:53:33.250535 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:53:33.250535 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:53:33.250535 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:53:33.250535 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:53:33.250535 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:53:33.250535 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:53:33.250535 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:53:33.250535 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:53:33.250535 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:53:33.250535 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 16:53:33.250535 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:53:33.253346 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 75/151] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:53:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:53:33.259417 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de3b808-2b04-47de-b5be-cfc2e9d195f7 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:33.417381 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de3b808-2b04-47de-b5be-cfc2e9d195f7 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:33.473863 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e78c085-aee4-4254-8714-652b8bef4288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:33.474491 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e78c085-aee4-4254-8714-652b8bef4288 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:33.474775 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e78c085-aee4-4254-8714-652b8bef4288 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:33.475082 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e78c085-aee4-4254-8714-652b8bef4288 None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:33.475763 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e78c085-aee4-4254-8714-652b8bef4288 None admin] RBAC: Adding query filter params (name=member) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:53:33.476071 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e78c085-aee4-4254-8714-652b8bef4288 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:33.477470 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e78c085-aee4-4254-8714-652b8bef4288 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:33.491319 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 77/152] 10.222.0.82 () {56 vars in 1101 bytes} [Thu Aug 18 16:53:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:33.496596 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20510ea5-d4be-4165-9581-7d68299dd1ca None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:33.634502 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20510ea5-d4be-4165-9581-7d68299dd1ca None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:33.678474 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264c5f0f-3a43-430b-966b-2348aab35850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:33.679494 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264c5f0f-3a43-430b-966b-2348aab35850 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:33.679980 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264c5f0f-3a43-430b-966b-2348aab35850 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:33.680445 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264c5f0f-3a43-430b-966b-2348aab35850 None admin] PATH_INFO: `/v3/users/3f70ad03afa246c393b5448fe7ee6860` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:33.681807 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264c5f0f-3a43-430b-966b-2348aab35850 None admin] RBAC: Authorizing `identity:get_user(user_id=3f70ad03afa246c393b5448fe7ee6860)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:33.683700 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264c5f0f-3a43-430b-966b-2348aab35850 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:33.685580 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 76/153] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 16:53:33 2022] GET /identity/v3/users/3f70ad03afa246c393b5448fe7ee6860 => generated 275 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:33.691790 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e78c085-aee4-4254-8714-652b8bef4288 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:33.742068 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:33.835749 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e78c085-aee4-4254-8714-652b8bef4288 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:33.851440 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d9328-413e-4768-9f5a-0fe3cef4e8e2 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:33.851868 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d9328-413e-4768-9f5a-0fe3cef4e8e2 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:33.852251 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d9328-413e-4768-9f5a-0fe3cef4e8e2 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:33.853833 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 77/154] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:53:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 16:53:33.859413 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a2397d-bfa3-4a8c-8604-4647b04dc7a5 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:33.859808 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a2397d-bfa3-4a8c-8604-4647b04dc7a5 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:33.860189 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a2397d-bfa3-4a8c-8604-4647b04dc7a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:33.876426 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a07fc7-d71b-4050-bec7-5ebdfdc7e268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:33.877354 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a07fc7-d71b-4050-bec7-5ebdfdc7e268 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:33.877978 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a07fc7-d71b-4050-bec7-5ebdfdc7e268 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:33.878626 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a07fc7-d71b-4050-bec7-5ebdfdc7e268 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:33.879529 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:33.901219 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-33a2397d-bfa3-4a8c-8604-4647b04dc7a5 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:53:33.902418 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-33a2397d-bfa3-4a8c-8604-4647b04dc7a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:53:34.002152 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 78/155] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:53:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:53:34.005418 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a07fc7-d71b-4050-bec7-5ebdfdc7e268 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:34.006939 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a07fc7-d71b-4050-bec7-5ebdfdc7e268 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:34.061602 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:34.139801 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 78/156] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:53:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 16:53:34.145745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a2397d-bfa3-4a8c-8604-4647b04dc7a5 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:34.266474 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3ab54f-6554-4fe8-a3af-b312faea2031 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:34.267098 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3ab54f-6554-4fe8-a3af-b312faea2031 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:34.267098 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3ab54f-6554-4fe8-a3af-b312faea2031 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:34.269337 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 79/157] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:53:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:53:34.271205 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a2397d-bfa3-4a8c-8604-4647b04dc7a5 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:34.292454 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc56611b-4d75-4a8d-bbf3-a136e6aaee07 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:34.292785 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc56611b-4d75-4a8d-bbf3-a136e6aaee07 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:34.292969 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc56611b-4d75-4a8d-bbf3-a136e6aaee07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:34.310439 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:34.314455 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1b5340-4c4f-4914-8eae-f438ac45c336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:34.315330 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b5340-4c4f-4914-8eae-f438ac45c336 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:34.315814 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b5340-4c4f-4914-8eae-f438ac45c336 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:34.316271 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b5340-4c4f-4914-8eae-f438ac45c336 None admin] PATH_INFO: `/v3/projects/62beed06c75a4725ae7834995533d0a0` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:34.320017 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-bc56611b-4d75-4a8d-bbf3-a136e6aaee07 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:53:34.320017 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-bc56611b-4d75-4a8d-bbf3-a136e6aaee07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:53:34.320971 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1b5340-4c4f-4914-8eae-f438ac45c336 None admin] RBAC: Authorizing `identity:get_project(project_id=62beed06c75a4725ae7834995533d0a0)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:34.327152 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1b5340-4c4f-4914-8eae-f438ac45c336 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:34.327152 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 79/158] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 16:53:34 2022] GET /identity/v3/projects/62beed06c75a4725ae7834995533d0a0 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:34.333245 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1b5340-4c4f-4914-8eae-f438ac45c336 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:34.433250 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 80/159] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:53:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:53:34.497502 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1b5340-4c4f-4914-8eae-f438ac45c336 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:34.536590 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f02ada7-30fd-421a-ab54-d6779fc12f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:34.537218 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f02ada7-30fd-421a-ab54-d6779fc12f66 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:34.537507 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f02ada7-30fd-421a-ab54-d6779fc12f66 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:34.537783 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f02ada7-30fd-421a-ab54-d6779fc12f66 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:34.538520 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f02ada7-30fd-421a-ab54-d6779fc12f66 None admin] RBAC: Adding query filter params (role.id=36d6d323d9e64e659591dbaea3081bcc, scope.project.id=62beed06c75a4725ae7834995533d0a0, user.id=3f70ad03afa246c393b5448fe7ee6860) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:53:34.538827 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f02ada7-30fd-421a-ab54-d6779fc12f66 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:34.540455 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f02ada7-30fd-421a-ab54-d6779fc12f66 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:34.551033 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 80/160] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 16:53:34 2022] GET /identity/v3/role_assignments?user.id=3f70ad03afa246c393b5448fe7ee6860&scope.project.id=62beed06c75a4725ae7834995533d0a0&role.id=36d6d323d9e64e659591dbaea3081bcc => generated 608 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:34.651317 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc56611b-4d75-4a8d-bbf3-a136e6aaee07 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:34.722142 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:34.767529 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc56611b-4d75-4a8d-bbf3-a136e6aaee07 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:34.820693 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fe0aa1-58b1-41de-b747-45c695d31e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:34.821268 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe0aa1-58b1-41de-b747-45c695d31e87 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:34.821551 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe0aa1-58b1-41de-b747-45c695d31e87 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:34.821826 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe0aa1-58b1-41de-b747-45c695d31e87 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:34.833414 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fe0aa1-58b1-41de-b747-45c695d31e87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:53:34.838389 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-f9fe0aa1-58b1-41de-b747-45c695d31e87 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 16:53:34.838389 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:53:34.838389 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:53:34.838389 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:53:34.838389 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:53:34.838389 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:53:34.838389 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:53:34.838389 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:53:34.838389 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:53:34.838389 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:53:34.838389 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:53:34.838389 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:53:34.838389 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:53:34.838389 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:53:34.838389 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:53:34.838389 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:53:34.839297 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:53:34.839297 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:53:34.839297 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:53:34.839297 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:53:34.839297 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:53:34.839297 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:53:34.839297 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:53:34.839297 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:53:34.839297 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:53:34.839297 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:53:34.839297 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:53:34.839297 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:53:34.839297 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:53:34.839297 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:53:34.839297 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:53:34.839297 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:53:34.840125 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:53:34.840125 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:53:34.840125 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:53:34.840125 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:53:34.840125 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:53:34.840125 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:53:34.840125 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:53:34.840125 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:53:34.840125 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:53:34.840125 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:53:34.840125 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 16:53:34.840125 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:53:34.840125 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 81/161] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 16:53:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:53:34.847438 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f02ada7-30fd-421a-ab54-d6779fc12f66 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:34.933192 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:34.955285 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fe0aa1-58b1-41de-b747-45c695d31e87 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:34.994548 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f02ada7-30fd-421a-ab54-d6779fc12f66 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:35.055915 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419b8f70-edcf-4a70-817a-9810ebfd2274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:35.056546 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b8f70-edcf-4a70-817a-9810ebfd2274 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:35.056834 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b8f70-edcf-4a70-817a-9810ebfd2274 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:35.057111 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b8f70-edcf-4a70-817a-9810ebfd2274 None admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:35.057781 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419b8f70-edcf-4a70-817a-9810ebfd2274 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:53:35.058089 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419b8f70-edcf-4a70-817a-9810ebfd2274 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:35.059526 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419b8f70-edcf-4a70-817a-9810ebfd2274 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:35.074100 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 81/162] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:53:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:53:35.077434 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8477f23-49f2-41af-b73f-c7bf36c406f2 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:35.078309 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8477f23-49f2-41af-b73f-c7bf36c406f2 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:35.081600 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8477f23-49f2-41af-b73f-c7bf36c406f2 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:35.082960 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 82/163] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:53:35 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:53:35.083705 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8477f23-49f2-41af-b73f-c7bf36c406f2 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:35.157041 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fe0aa1-58b1-41de-b747-45c695d31e87 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:35.209612 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:35.229500 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32813daf-a379-404e-9aca-ebac94c37178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:35.230166 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32813daf-a379-404e-9aca-ebac94c37178 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:35.230430 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32813daf-a379-404e-9aca-ebac94c37178 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:35.230701 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32813daf-a379-404e-9aca-ebac94c37178 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:35.239742 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8477f23-49f2-41af-b73f-c7bf36c406f2 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:35.260024 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32813daf-a379-404e-9aca-ebac94c37178 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:53:35.270286 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-32813daf-a379-404e-9aca-ebac94c37178 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 16:53:35.270286 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:53:35.270286 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:53:35.270286 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:53:35.270286 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:53:35.270286 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:53:35.270286 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:53:35.270286 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:53:35.270286 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:53:35.270286 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:53:35.270286 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:53:35.270286 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:53:35.270286 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:53:35.270286 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:53:35.270286 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:53:35.270286 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:53:35.271598 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:53:35.271598 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:53:35.271598 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:53:35.271598 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:53:35.271598 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:53:35.271598 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:53:35.271598 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:53:35.271598 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:53:35.271598 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:53:35.271598 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:53:35.271598 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:53:35.271598 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:53:35.271598 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:53:35.271598 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:53:35.271598 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:53:35.271598 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:53:35.272360 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:53:35.272360 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:53:35.272360 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:53:35.272360 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:53:35.272360 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:53:35.272360 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:53:35.272360 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:53:35.272360 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:53:35.272360 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:53:35.272360 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:53:35.272360 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 16:53:35.272360 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:53:35.277034 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 82/164] 10.222.0.82 () {56 vars in 1114 bytes} [Thu Aug 18 16:53:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:53:35.278596 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e360d2-dc4e-46d2-809e-1ed91e501acc None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:35.279837 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e360d2-dc4e-46d2-809e-1ed91e501acc None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:35.281135 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e360d2-dc4e-46d2-809e-1ed91e501acc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:35.287902 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:35.299699 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261fe002-f796-43a8-9bc7-32f83d67f4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:35.306067 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261fe002-f796-43a8-9bc7-32f83d67f4d0 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:35.306511 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261fe002-f796-43a8-9bc7-32f83d67f4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:35.306744 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261fe002-f796-43a8-9bc7-32f83d67f4d0 None admin] PATH_INFO: `/v3/users/329c213a845c4ab2a0642e5a7dbd4b18` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:35.308174 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261fe002-f796-43a8-9bc7-32f83d67f4d0 None admin] RBAC: Authorizing `identity:get_user(user_id=329c213a845c4ab2a0642e5a7dbd4b18)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:35.309669 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261fe002-f796-43a8-9bc7-32f83d67f4d0 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:35.313823 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 83/165] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 16:53:35 2022] GET /identity/v3/users/329c213a845c4ab2a0642e5a7dbd4b18 => generated 247 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:35.316747 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261fe002-f796-43a8-9bc7-32f83d67f4d0 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:35.322216 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-77e360d2-dc4e-46d2-809e-1ed91e501acc None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:53:35.323488 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-77e360d2-dc4e-46d2-809e-1ed91e501acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:53:35.409182 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 83/166] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:53:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:53:35.410096 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e360d2-dc4e-46d2-809e-1ed91e501acc None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:35.520615 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e360d2-dc4e-46d2-809e-1ed91e501acc None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:35.834866 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:35.834866 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdc797b-c76b-43d2-8bfb-eb1886a55df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:35.834866 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdc797b-c76b-43d2-8bfb-eb1886a55df6 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:35.834866 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdc797b-c76b-43d2-8bfb-eb1886a55df6 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:35.835762 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdc797b-c76b-43d2-8bfb-eb1886a55df6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:35.835887 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261fe002-f796-43a8-9bc7-32f83d67f4d0 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:35.835887 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf32ab9b-3810-4be1-a54b-f321c2b14b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:35.835887 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf32ab9b-3810-4be1-a54b-f321c2b14b02 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:35.836099 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdc797b-c76b-43d2-8bfb-eb1886a55df6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:35.836170 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf32ab9b-3810-4be1-a54b-f321c2b14b02 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:35.836170 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf32ab9b-3810-4be1-a54b-f321c2b14b02 None admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:35.836170 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf32ab9b-3810-4be1-a54b-f321c2b14b02 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:53:35.836170 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf32ab9b-3810-4be1-a54b-f321c2b14b02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:35.836414 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdc797b-c76b-43d2-8bfb-eb1886a55df6 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:35.836483 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf32ab9b-3810-4be1-a54b-f321c2b14b02 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:35.836483 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 84/167] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 16:53:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:35.836483 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf32ab9b-3810-4be1-a54b-f321c2b14b02 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:35.836483 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:35.836739 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 84/168] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:53:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 16:53:35.836739 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdc797b-c76b-43d2-8bfb-eb1886a55df6 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:35.836739 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:35.836925 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf32ab9b-3810-4be1-a54b-f321c2b14b02 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:35.844811 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37090e06-e04f-4b1f-bbad-0a35e8bc7da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:35.846172 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37090e06-e04f-4b1f-bbad-0a35e8bc7da3 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:35.847926 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37090e06-e04f-4b1f-bbad-0a35e8bc7da3 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:35.847926 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37090e06-e04f-4b1f-bbad-0a35e8bc7da3 None admin] PATH_INFO: `/v3/users/3f70ad03afa246c393b5448fe7ee6860` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:35.847926 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37090e06-e04f-4b1f-bbad-0a35e8bc7da3 None admin] RBAC: Authorizing `identity:get_user(user_id=3f70ad03afa246c393b5448fe7ee6860)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:35.858928 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37090e06-e04f-4b1f-bbad-0a35e8bc7da3 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:35.859508 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 85/169] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 16:53:35 2022] GET /identity/v3/users/3f70ad03afa246c393b5448fe7ee6860 => generated 275 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:53:35.860585 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37090e06-e04f-4b1f-bbad-0a35e8bc7da3 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:35.901544 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdc797b-c76b-43d2-8bfb-eb1886a55df6 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:35.938386 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ade1f02-196e-4f6a-a321-180efc4ae5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:35.939869 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade1f02-196e-4f6a-a321-180efc4ae5af None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:35.940049 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade1f02-196e-4f6a-a321-180efc4ae5af None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:35.946857 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade1f02-196e-4f6a-a321-180efc4ae5af None admin] PATH_INFO: `/v3/projects/62beed06c75a4725ae7834995533d0a0` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:35.946857 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ade1f02-196e-4f6a-a321-180efc4ae5af None admin] RBAC: Authorizing `identity:get_project(project_id=62beed06c75a4725ae7834995533d0a0)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:35.946857 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ade1f02-196e-4f6a-a321-180efc4ae5af None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:35.946857 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 85/170] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 16:53:35 2022] GET /identity/v3/projects/62beed06c75a4725ae7834995533d0a0 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:35.952550 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ade1f02-196e-4f6a-a321-180efc4ae5af None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:36.011264 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37090e06-e04f-4b1f-bbad-0a35e8bc7da3 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:36.040218 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:36.051415 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:36.059561 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ade1f02-196e-4f6a-a321-180efc4ae5af None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:36.072184 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515edda4-3415-4045-9c81-f047ae788946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:36.072814 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515edda4-3415-4045-9c81-f047ae788946 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:36.073101 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515edda4-3415-4045-9c81-f047ae788946 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:36.073381 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515edda4-3415-4045-9c81-f047ae788946 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:36.118639 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b4e7b4-b386-4d4c-b9a1-3f859ee6f57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:36.118639 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b4e7b4-b386-4d4c-b9a1-3f859ee6f57b None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:36.118639 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b4e7b4-b386-4d4c-b9a1-3f859ee6f57b None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:36.118639 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b4e7b4-b386-4d4c-b9a1-3f859ee6f57b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:36.119165 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b4e7b4-b386-4d4c-b9a1-3f859ee6f57b None admin] RBAC: Adding query filter params (role.id=2c6966cfb10f4d72b0242e9fb5e13b17, scope.project.id=62beed06c75a4725ae7834995533d0a0, user.id=329c213a845c4ab2a0642e5a7dbd4b18) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:53:36.119165 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b4e7b4-b386-4d4c-b9a1-3f859ee6f57b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:36.119165 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b4e7b4-b386-4d4c-b9a1-3f859ee6f57b None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:36.119165 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 86/171] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 16:53:35 2022] GET /identity/v3/role_assignments?user.id=329c213a845c4ab2a0642e5a7dbd4b18&scope.project.id=62beed06c75a4725ae7834995533d0a0&role.id=2c6966cfb10f4d72b0242e9fb5e13b17 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:53:36.119165 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b4e7b4-b386-4d4c-b9a1-3f859ee6f57b None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:36.204137 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515edda4-3415-4045-9c81-f047ae788946 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:36.206239 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515edda4-3415-4045-9c81-f047ae788946 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:36.366659 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 86/172] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:53:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 16:53:36.368069 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970fd235-ab04-42fc-86af-d92908d77ef3 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:36.368363 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970fd235-ab04-42fc-86af-d92908d77ef3 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:36.368636 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970fd235-ab04-42fc-86af-d92908d77ef3 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:36.370232 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 87/173] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:53:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 16:53:36.373658 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970fd235-ab04-42fc-86af-d92908d77ef3 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:36.405002 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:36.433849 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b4e7b4-b386-4d4c-b9a1-3f859ee6f57b None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:36.484057 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df546517-9611-42dc-8e1c-1c81eb68e007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:36.484977 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df546517-9611-42dc-8e1c-1c81eb68e007 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:36.485263 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df546517-9611-42dc-8e1c-1c81eb68e007 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:36.485537 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df546517-9611-42dc-8e1c-1c81eb68e007 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:36.496635 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970fd235-ab04-42fc-86af-d92908d77ef3 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:36.498812 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df546517-9611-42dc-8e1c-1c81eb68e007 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:53:36.505534 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-df546517-9611-42dc-8e1c-1c81eb68e007 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 16:53:36.505534 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:53:36.505534 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:53:36.505534 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:53:36.505534 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:53:36.505534 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:53:36.505534 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:53:36.505534 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:53:36.505534 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:53:36.505534 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:53:36.505534 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:53:36.505534 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:53:36.505534 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:53:36.505534 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:53:36.505534 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:53:36.505534 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:53:36.506395 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:53:36.506395 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:53:36.506395 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:53:36.506395 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:53:36.506395 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:53:36.506395 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:53:36.506395 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:53:36.506395 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:53:36.506395 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:53:36.506395 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:53:36.506395 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:53:36.506395 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:53:36.506395 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:53:36.506395 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:53:36.506395 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:53:36.506395 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:53:36.507290 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:53:36.507290 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:53:36.507290 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:53:36.507290 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:53:36.507290 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:53:36.507290 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:53:36.507290 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:53:36.507290 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:53:36.507290 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:53:36.507290 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:53:36.507290 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 16:53:36.507290 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:53:36.507290 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 87/174] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:53:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 399 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:53:36.508848 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d441e0e-ea5f-4fbc-b711-c50bcc7be368 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:36.509141 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d441e0e-ea5f-4fbc-b711-c50bcc7be368 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:36.509425 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d441e0e-ea5f-4fbc-b711-c50bcc7be368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:36.531282 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:36.536233 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-1d441e0e-ea5f-4fbc-b711-c50bcc7be368 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:53:36.537630 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-1d441e0e-ea5f-4fbc-b711-c50bcc7be368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:53:36.545009 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935c47c0-b329-42a2-a69f-082303bd485e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:36.552790 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c47c0-b329-42a2-a69f-082303bd485e None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:36.552790 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c47c0-b329-42a2-a69f-082303bd485e None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:36.552790 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c47c0-b329-42a2-a69f-082303bd485e None admin] PATH_INFO: `/v3/projects/62beed06c75a4725ae7834995533d0a0` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:36.552790 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935c47c0-b329-42a2-a69f-082303bd485e None admin] RBAC: Authorizing `identity:get_project(project_id=62beed06c75a4725ae7834995533d0a0)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:36.552790 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935c47c0-b329-42a2-a69f-082303bd485e None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:36.552790 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 88/175] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 16:53:36 2022] GET /identity/v3/projects/62beed06c75a4725ae7834995533d0a0 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:53:36.553203 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df2e740-a3fb-4169-95f1-9fbb418ab881 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:36.553508 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df2e740-a3fb-4169-95f1-9fbb418ab881 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:36.554061 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df2e740-a3fb-4169-95f1-9fbb418ab881 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:36.555529 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 89/176] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:53:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:53:36.556434 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df2e740-a3fb-4169-95f1-9fbb418ab881 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:36.611665 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 88/177] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:53:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:53:36.613056 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade04260-4061-4016-8f33-93466a90eb4d None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:36.613350 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade04260-4061-4016-8f33-93466a90eb4d None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:36.613628 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade04260-4061-4016-8f33-93466a90eb4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:36.640974 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-ade04260-4061-4016-8f33-93466a90eb4d None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:53:36.642619 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-ade04260-4061-4016-8f33-93466a90eb4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:53:36.698535 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:36.717316 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df2e740-a3fb-4169-95f1-9fbb418ab881 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:36.781809 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 89/178] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:53:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:53:36.782374 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade04260-4061-4016-8f33-93466a90eb4d None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:36.794561 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f272718-e527-41f6-9c1d-b7fa59073287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:36.795213 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f272718-e527-41f6-9c1d-b7fa59073287 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:36.795497 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f272718-e527-41f6-9c1d-b7fa59073287 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:36.796298 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f272718-e527-41f6-9c1d-b7fa59073287 None admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:36.796983 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f272718-e527-41f6-9c1d-b7fa59073287 None admin] RBAC: Adding query filter params (name=member) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:53:36.797340 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f272718-e527-41f6-9c1d-b7fa59073287 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:36.799877 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f272718-e527-41f6-9c1d-b7fa59073287 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:36.817453 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 90/179] 10.222.0.82 () {56 vars in 1101 bytes} [Thu Aug 18 16:53:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:36.823440 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f272718-e527-41f6-9c1d-b7fa59073287 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:36.886932 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:36.925600 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade04260-4061-4016-8f33-93466a90eb4d None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:36.960743 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f272718-e527-41f6-9c1d-b7fa59073287 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:36.962524 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41e91a1-32ae-47a8-9612-4ab316ab5cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:36.963170 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41e91a1-32ae-47a8-9612-4ab316ab5cbe None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:36.963446 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41e91a1-32ae-47a8-9612-4ab316ab5cbe None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:36.963717 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41e91a1-32ae-47a8-9612-4ab316ab5cbe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:36.964444 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41e91a1-32ae-47a8-9612-4ab316ab5cbe None admin] RBAC: Adding query filter params (role.id=f41f6f29d9de4b1985c8c8c14f002cf2, scope.project.id=62beed06c75a4725ae7834995533d0a0, user.id=3f70ad03afa246c393b5448fe7ee6860) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:53:36.964752 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41e91a1-32ae-47a8-9612-4ab316ab5cbe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:36.966216 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41e91a1-32ae-47a8-9612-4ab316ab5cbe None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:36.973153 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 90/180] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 16:53:36 2022] GET /identity/v3/role_assignments?user.id=3f70ad03afa246c393b5448fe7ee6860&scope.project.id=62beed06c75a4725ae7834995533d0a0&role.id=f41f6f29d9de4b1985c8c8c14f002cf2 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:36.974147 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c0f5e5-1507-411c-af44-18bebd428cba None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:36.974432 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c0f5e5-1507-411c-af44-18bebd428cba None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:36.974703 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c0f5e5-1507-411c-af44-18bebd428cba None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:36.976094 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 91/181] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:53:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:53:36.976972 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ecb8e-9d8d-4ca4-a6f2-c5f833d2017d None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:36.977255 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ecb8e-9d8d-4ca4-a6f2-c5f833d2017d None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:36.977530 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ecb8e-9d8d-4ca4-a6f2-c5f833d2017d None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:36.978816 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 92/182] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:53:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:53:36.979565 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ecb8e-9d8d-4ca4-a6f2-c5f833d2017d None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:37.009386 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6f167f-4d86-4d85-a4f7-448ff3e38756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:37.010450 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f167f-4d86-4d85-a4f7-448ff3e38756 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:37.011273 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f167f-4d86-4d85-a4f7-448ff3e38756 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:37.012288 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f167f-4d86-4d85-a4f7-448ff3e38756 None admin] PATH_INFO: `/v3/users/3f70ad03afa246c393b5448fe7ee6860` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:37.014431 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6f167f-4d86-4d85-a4f7-448ff3e38756 None admin] RBAC: Authorizing `identity:get_user(user_id=3f70ad03afa246c393b5448fe7ee6860)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:37.017098 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6f167f-4d86-4d85-a4f7-448ff3e38756 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:37.025382 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 91/183] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 16:53:36 2022] GET /identity/v3/users/3f70ad03afa246c393b5448fe7ee6860 => generated 275 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:53:37.025382 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a7a10d-0187-446a-875c-7546b519fe6f None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:37.025382 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a7a10d-0187-446a-875c-7546b519fe6f None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:37.025382 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a7a10d-0187-446a-875c-7546b519fe6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:37.044268 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-f6a7a10d-0187-446a-875c-7546b519fe6f None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:53:37.045508 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-f6a7a10d-0187-446a-875c-7546b519fe6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:53:37.065810 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:37.125895 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 92/184] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:53:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:53:37.126095 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ecb8e-9d8d-4ca4-a6f2-c5f833d2017d None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:37.127169 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f42aa-6e57-4b11-b81a-e1b00ce204dc None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:37.127464 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f42aa-6e57-4b11-b81a-e1b00ce204dc None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:37.127738 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f42aa-6e57-4b11-b81a-e1b00ce204dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:37.151635 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-bd9f42aa-6e57-4b11-b81a-e1b00ce204dc None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:53:37.152785 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-bd9f42aa-6e57-4b11-b81a-e1b00ce204dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:53:37.163220 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:37.170664 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8925b1f6-5ff8-45f8-b320-d1d784169f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:37.171446 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8925b1f6-5ff8-45f8-b320-d1d784169f7e None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:37.171723 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8925b1f6-5ff8-45f8-b320-d1d784169f7e None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:37.171998 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8925b1f6-5ff8-45f8-b320-d1d784169f7e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:37.190188 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8925b1f6-5ff8-45f8-b320-d1d784169f7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:53:37.190188 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-8925b1f6-5ff8-45f8-b320-d1d784169f7e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 16:53:37.190188 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:53:37.190188 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:53:37.190188 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:53:37.190188 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:53:37.190188 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:53:37.190188 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:53:37.190188 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:53:37.190188 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:53:37.190188 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:53:37.190188 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:53:37.190188 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:53:37.190188 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:53:37.191038 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:53:37.191038 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:53:37.191038 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:53:37.191038 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:53:37.191038 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:53:37.191038 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:53:37.191038 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:53:37.191038 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:53:37.191038 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:53:37.191038 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:53:37.191038 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:53:37.191038 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:53:37.191038 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:53:37.191038 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:53:37.191038 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:53:37.191038 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:53:37.191038 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:53:37.191859 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:53:37.191859 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:53:37.191859 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:53:37.191859 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:53:37.191859 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:53:37.191859 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:53:37.191859 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:53:37.191859 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:53:37.191859 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:53:37.191859 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:53:37.191859 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:53:37.191859 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:53:37.191859 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 16:53:37.191859 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:53:37.192537 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 93/185] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:53:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:53:37.193425 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8925b1f6-5ff8-45f8-b320-d1d784169f7e None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:37.240122 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 93/186] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:53:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:53:37.241057 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9f42aa-6e57-4b11-b81a-e1b00ce204dc None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:37.325942 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8925b1f6-5ff8-45f8-b320-d1d784169f7e None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:37.335250 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:37.352927 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9f42aa-6e57-4b11-b81a-e1b00ce204dc None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:37.363623 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f2904b-d02c-4fcd-8a8f-73013484d8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:37.364277 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f2904b-d02c-4fcd-8a8f-73013484d8af None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:37.364555 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f2904b-d02c-4fcd-8a8f-73013484d8af None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:37.364828 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f2904b-d02c-4fcd-8a8f-73013484d8af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:37.387927 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8104811-91ea-4ec2-b761-a51d3d271d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:37.388579 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8104811-91ea-4ec2-b761-a51d3d271d8c None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:37.388840 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8104811-91ea-4ec2-b761-a51d3d271d8c None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:37.389111 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8104811-91ea-4ec2-b761-a51d3d271d8c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:37.400123 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8104811-91ea-4ec2-b761-a51d3d271d8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:53:37.404856 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-d8104811-91ea-4ec2-b761-a51d3d271d8c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 16:53:37.404856 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:53:37.404856 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:53:37.404856 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:53:37.404856 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:53:37.404856 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:53:37.404856 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:53:37.404856 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:53:37.404856 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:53:37.404856 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:53:37.404856 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:53:37.404856 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:53:37.404856 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:53:37.404856 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:53:37.404856 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:53:37.404856 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:53:37.405722 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:53:37.405722 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:53:37.405722 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:53:37.405722 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:53:37.405722 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:53:37.405722 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:53:37.405722 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:53:37.405722 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:53:37.405722 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:53:37.405722 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:53:37.405722 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:53:37.405722 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:53:37.405722 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:53:37.405722 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:53:37.405722 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:53:37.405722 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:53:37.406480 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:53:37.406480 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:53:37.406480 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:53:37.406480 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:53:37.406480 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:53:37.406480 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:53:37.406480 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:53:37.406480 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:53:37.406480 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:53:37.406480 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:53:37.406480 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 16:53:37.406480 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:53:37.406480 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 94/187] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 16:53:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:53:37.408461 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8104811-91ea-4ec2-b761-a51d3d271d8c None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:37.432272 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:37.453324 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f2904b-d02c-4fcd-8a8f-73013484d8af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:37.453324 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f2904b-d02c-4fcd-8a8f-73013484d8af None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:37.519484 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8104811-91ea-4ec2-b761-a51d3d271d8c None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:37.548737 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 94/188] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:53:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 16:53:37.549617 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f2904b-d02c-4fcd-8a8f-73013484d8af None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:37.559304 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2d8bf0-a38a-489f-9ae9-2b323bba16e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:37.559915 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2d8bf0-a38a-489f-9ae9-2b323bba16e1 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:37.560200 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2d8bf0-a38a-489f-9ae9-2b323bba16e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:37.560472 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2d8bf0-a38a-489f-9ae9-2b323bba16e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:37.561131 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2d8bf0-a38a-489f-9ae9-2b323bba16e1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:53:37.561432 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2d8bf0-a38a-489f-9ae9-2b323bba16e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:37.562853 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2d8bf0-a38a-489f-9ae9-2b323bba16e1 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:37.571205 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 95/189] 10.222.0.82 () {56 vars in 1101 bytes} [Thu Aug 18 16:53:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:37.572017 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2d8bf0-a38a-489f-9ae9-2b323bba16e1 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:37.576849 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:37.637654 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:37.647345 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f2904b-d02c-4fcd-8a8f-73013484d8af None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:37.671914 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2d8bf0-a38a-489f-9ae9-2b323bba16e1 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:37.680440 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b1c4a8-25d0-4d43-b74f-2b71995f3976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:37.681069 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b1c4a8-25d0-4d43-b74f-2b71995f3976 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:37.681346 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b1c4a8-25d0-4d43-b74f-2b71995f3976 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:37.681618 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b1c4a8-25d0-4d43-b74f-2b71995f3976 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:37.691788 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b1c4a8-25d0-4d43-b74f-2b71995f3976 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:53:37.696481 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-a5b1c4a8-25d0-4d43-b74f-2b71995f3976 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 16:53:37.696481 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:53:37.696481 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:53:37.696481 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:53:37.696481 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:53:37.696481 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:53:37.696481 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:53:37.696481 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:53:37.696481 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:53:37.696481 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:53:37.696481 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:53:37.696481 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:53:37.696481 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:53:37.696481 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:53:37.696481 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:53:37.696481 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:53:37.697491 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:53:37.697491 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:53:37.697491 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:53:37.697491 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:53:37.697491 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:53:37.697491 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:53:37.697491 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:53:37.697491 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:53:37.697491 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:53:37.697491 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:53:37.697491 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:53:37.697491 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:53:37.697491 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:53:37.697491 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:53:37.697491 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:53:37.697491 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:53:37.698476 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:53:37.698476 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:53:37.698476 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:53:37.698476 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:53:37.698476 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:53:37.698476 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:53:37.698476 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:53:37.698476 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:53:37.698476 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:53:37.698476 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:53:37.698476 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 16:53:37.698476 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:53:37.698476 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 95/190] 10.222.0.82 () {56 vars in 1114 bytes} [Thu Aug 18 16:53:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:53:37.698476 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b1c4a8-25d0-4d43-b74f-2b71995f3976 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:37.709733 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd2a6da-7355-443f-a6e4-d227f084efaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:37.710284 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd2a6da-7355-443f-a6e4-d227f084efaa None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:37.710568 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd2a6da-7355-443f-a6e4-d227f084efaa None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:37.710840 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd2a6da-7355-443f-a6e4-d227f084efaa None admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:37.711537 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd2a6da-7355-443f-a6e4-d227f084efaa None admin] RBAC: Adding query filter params (name=admin) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:53:37.711871 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd2a6da-7355-443f-a6e4-d227f084efaa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:37.713261 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd2a6da-7355-443f-a6e4-d227f084efaa None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:37.721480 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 96/191] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:53:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:37.721480 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd2a6da-7355-443f-a6e4-d227f084efaa None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:37.790675 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:37.802700 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b1c4a8-25d0-4d43-b74f-2b71995f3976 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:37.825396 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd2a6da-7355-443f-a6e4-d227f084efaa None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:37.836835 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d0a8dd-0228-4490-ad12-0f5a931f963c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:37.837470 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0a8dd-0228-4490-ad12-0f5a931f963c None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:37.837752 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0a8dd-0228-4490-ad12-0f5a931f963c None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:37.838025 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0a8dd-0228-4490-ad12-0f5a931f963c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:37.839168 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d0a8dd-0228-4490-ad12-0f5a931f963c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:37.840630 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d0a8dd-0228-4490-ad12-0f5a931f963c None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:37.842346 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 96/192] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:53:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:37.842953 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d0a8dd-0228-4490-ad12-0f5a931f963c None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:37.861572 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:37.863714 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13078152-ba27-4924-b6c9-e7a1a8263da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:37.864347 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13078152-ba27-4924-b6c9-e7a1a8263da1 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:37.864629 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13078152-ba27-4924-b6c9-e7a1a8263da1 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:37.864905 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13078152-ba27-4924-b6c9-e7a1a8263da1 None admin] PATH_INFO: `/v3/projects/244b82f9ef034b6bb3d250becacf818d` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:37.866020 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13078152-ba27-4924-b6c9-e7a1a8263da1 None admin] RBAC: Authorizing `identity:get_project(project_id=244b82f9ef034b6bb3d250becacf818d)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:37.867506 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13078152-ba27-4924-b6c9-e7a1a8263da1 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:37.869111 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 97/193] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 16:53:37 2022] GET /identity/v3/projects/244b82f9ef034b6bb3d250becacf818d => generated 312 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:37.876863 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13078152-ba27-4924-b6c9-e7a1a8263da1 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:37.944766 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d0a8dd-0228-4490-ad12-0f5a931f963c None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:37.970547 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13078152-ba27-4924-b6c9-e7a1a8263da1 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:37.977746 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a80ca92-c576-49b6-90a5-f7ba532d48fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:37.978367 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a80ca92-c576-49b6-90a5-f7ba532d48fc None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:37.978646 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a80ca92-c576-49b6-90a5-f7ba532d48fc None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:37.978936 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a80ca92-c576-49b6-90a5-f7ba532d48fc None admin] PATH_INFO: `/v3/users/f601719176bc414ab763d3f1fd6662d8` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:37.980052 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a80ca92-c576-49b6-90a5-f7ba532d48fc None admin] RBAC: Authorizing `identity:get_user(user_id=f601719176bc414ab763d3f1fd6662d8)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:37.981488 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a80ca92-c576-49b6-90a5-f7ba532d48fc None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:37.983145 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 97/194] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 16:53:37 2022] GET /identity/v3/users/f601719176bc414ab763d3f1fd6662d8 => generated 289 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:37.983907 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a80ca92-c576-49b6-90a5-f7ba532d48fc None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:38.005430 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090e3268-768a-42c2-9500-19377bec2966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:38.006051 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e3268-768a-42c2-9500-19377bec2966 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:38.006339 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e3268-768a-42c2-9500-19377bec2966 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:38.006617 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e3268-768a-42c2-9500-19377bec2966 None admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:38.007312 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090e3268-768a-42c2-9500-19377bec2966 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:53:38.007637 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090e3268-768a-42c2-9500-19377bec2966 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:38.009042 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090e3268-768a-42c2-9500-19377bec2966 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:38.010435 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:38.018514 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 98/195] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 16:53:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:38.019319 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090e3268-768a-42c2-9500-19377bec2966 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:38.073693 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:38.088114 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a80ca92-c576-49b6-90a5-f7ba532d48fc None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:38.118143 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090e3268-768a-42c2-9500-19377bec2966 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:38.122744 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11ab093-37ca-4cf5-a78d-7669a79068e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:38.123404 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ab093-37ca-4cf5-a78d-7669a79068e4 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:38.123686 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ab093-37ca-4cf5-a78d-7669a79068e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:38.123960 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ab093-37ca-4cf5-a78d-7669a79068e4 None admin] PATH_INFO: `/v3/users/1301612ad66a4086a33c4b0b0ab7e53a` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:38.138060 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11ab093-37ca-4cf5-a78d-7669a79068e4 None admin] RBAC: Authorizing `identity:get_user(user_id=1301612ad66a4086a33c4b0b0ab7e53a)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:38.139569 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11ab093-37ca-4cf5-a78d-7669a79068e4 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:38.145882 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 98/196] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 16:53:37 2022] GET /identity/v3/users/1301612ad66a4086a33c4b0b0ab7e53a => generated 283 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:38.145882 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11ab093-37ca-4cf5-a78d-7669a79068e4 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:38.152811 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c7f72e-dd53-40e2-8c90-c5cd27df3782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:38.153420 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c7f72e-dd53-40e2-8c90-c5cd27df3782 None admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:38.153706 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c7f72e-dd53-40e2-8c90-c5cd27df3782 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:38.153981 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c7f72e-dd53-40e2-8c90-c5cd27df3782 None admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:38.154617 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c7f72e-dd53-40e2-8c90-c5cd27df3782 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:38.156110 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c7f72e-dd53-40e2-8c90-c5cd27df3782 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:38.239287 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11ab093-37ca-4cf5-a78d-7669a79068e4 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:38.239613 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 99/197] 10.222.0.82 () {60 vars in 1132 bytes} [Thu Aug 18 16:53:38 2022] POST /identity/v3/users => generated 297 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 16:53:38.240442 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c7f72e-dd53-40e2-8c90-c5cd27df3782 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:38.285267 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea15834f-382e-4a87-818b-bdcae7e9914b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:38.285903 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15834f-382e-4a87-818b-bdcae7e9914b None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:38.286280 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15834f-382e-4a87-818b-bdcae7e9914b None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:38.286468 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15834f-382e-4a87-818b-bdcae7e9914b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:38.287240 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea15834f-382e-4a87-818b-bdcae7e9914b None admin] RBAC: Adding query filter params (role.id=36d6d323d9e64e659591dbaea3081bcc, scope.project.id=244b82f9ef034b6bb3d250becacf818d, user.id=3f70ad03afa246c393b5448fe7ee6860) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:53:38.287557 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea15834f-382e-4a87-818b-bdcae7e9914b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:38.289056 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea15834f-382e-4a87-818b-bdcae7e9914b None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:38.295753 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 99/198] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 16:53:38 2022] GET /identity/v3/role_assignments?user.id=3f70ad03afa246c393b5448fe7ee6860&scope.project.id=244b82f9ef034b6bb3d250becacf818d&role.id=36d6d323d9e64e659591dbaea3081bcc => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:38.307359 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea15834f-382e-4a87-818b-bdcae7e9914b None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:38.322660 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:38.323430 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:38.386088 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c7f72e-dd53-40e2-8c90-c5cd27df3782 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:38.435919 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb86f44a-ddc9-4d23-9eee-7190f3d6c70b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:38.436562 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb86f44a-ddc9-4d23-9eee-7190f3d6c70b None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:38.436839 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb86f44a-ddc9-4d23-9eee-7190f3d6c70b None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:38.437121 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb86f44a-ddc9-4d23-9eee-7190f3d6c70b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:38.451093 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea15834f-382e-4a87-818b-bdcae7e9914b None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:38.497609 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1dd74b-2a81-4c55-9d86-1317978dc87b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:38.498203 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1dd74b-2a81-4c55-9d86-1317978dc87b None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:38.498442 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1dd74b-2a81-4c55-9d86-1317978dc87b None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:38.498714 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1dd74b-2a81-4c55-9d86-1317978dc87b None admin] PATH_INFO: `/v3/users/1301612ad66a4086a33c4b0b0ab7e53a` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:38.499901 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1dd74b-2a81-4c55-9d86-1317978dc87b None admin] RBAC: Authorizing `identity:get_user(user_id=1301612ad66a4086a33c4b0b0ab7e53a)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:38.501987 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1dd74b-2a81-4c55-9d86-1317978dc87b None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:38.503848 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 100/199] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 16:53:38 2022] GET /identity/v3/users/1301612ad66a4086a33c4b0b0ab7e53a => generated 283 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:38.504618 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1dd74b-2a81-4c55-9d86-1317978dc87b None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:38.526151 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb86f44a-ddc9-4d23-9eee-7190f3d6c70b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:38.527696 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb86f44a-ddc9-4d23-9eee-7190f3d6c70b None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:38.594322 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:38.614026 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:38.640365 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1dd74b-2a81-4c55-9d86-1317978dc87b None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:38.653807 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 100/200] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:53:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 16:53:38.654679 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb86f44a-ddc9-4d23-9eee-7190f3d6c70b None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:38.681818 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ceb366-4692-48c6-88d9-6c672077e0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:38.682429 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ceb366-4692-48c6-88d9-6c672077e0d7 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:38.682705 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ceb366-4692-48c6-88d9-6c672077e0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:38.684545 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ceb366-4692-48c6-88d9-6c672077e0d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:38.761235 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ceb366-4692-48c6-88d9-6c672077e0d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:38.762627 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ceb366-4692-48c6-88d9-6c672077e0d7 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:38.778382 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb86f44a-ddc9-4d23-9eee-7190f3d6c70b None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:38.826517 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7568eb-b76d-47d4-9c6c-67271b8af861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:38.827171 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7568eb-b76d-47d4-9c6c-67271b8af861 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:38.827850 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7568eb-b76d-47d4-9c6c-67271b8af861 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:38.828134 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7568eb-b76d-47d4-9c6c-67271b8af861 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:38.854702 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:38.863175 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:38.897283 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 101/201] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:53:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 16:53:38.898636 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ceb366-4692-48c6-88d9-6c672077e0d7 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:38.907117 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7568eb-b76d-47d4-9c6c-67271b8af861 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:38.909459 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7568eb-b76d-47d4-9c6c-67271b8af861 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:39.020442 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ceb366-4692-48c6-88d9-6c672077e0d7 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:39.030201 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 101/202] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:53:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 16:53:39.030994 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7568eb-b76d-47d4-9c6c-67271b8af861 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:39.062172 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f175de29-d37d-4bea-8c79-c420f713db33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:39.063202 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f175de29-d37d-4bea-8c79-c420f713db33 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:39.064488 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f175de29-d37d-4bea-8c79-c420f713db33 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:39.065167 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f175de29-d37d-4bea-8c79-c420f713db33 None admin] PATH_INFO: `/v3/projects/62beed06c75a4725ae7834995533d0a0` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:39.066484 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f175de29-d37d-4bea-8c79-c420f713db33 None admin] RBAC: Authorizing `identity:get_project(project_id=62beed06c75a4725ae7834995533d0a0)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:39.068242 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f175de29-d37d-4bea-8c79-c420f713db33 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:39.070160 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 102/203] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 16:53:38 2022] GET /identity/v3/projects/62beed06c75a4725ae7834995533d0a0 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:39.071172 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f175de29-d37d-4bea-8c79-c420f713db33 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:39.088520 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:39.122746 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:39.139184 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7568eb-b76d-47d4-9c6c-67271b8af861 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:39.175694 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159e41df-e155-4ccb-a9a0-d461f043a07b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:39.176486 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159e41df-e155-4ccb-a9a0-d461f043a07b None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:39.176843 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159e41df-e155-4ccb-a9a0-d461f043a07b None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:39.177224 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159e41df-e155-4ccb-a9a0-d461f043a07b None admin] PATH_INFO: `/v3/projects/d5b629fedcc945bf8c55bda4ceedc4ae` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:39.178436 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159e41df-e155-4ccb-a9a0-d461f043a07b None admin] RBAC: Authorizing `identity:get_project(project_id=d5b629fedcc945bf8c55bda4ceedc4ae)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:39.180024 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159e41df-e155-4ccb-a9a0-d461f043a07b None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:39.181807 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 102/204] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 16:53:39 2022] GET /identity/v3/projects/d5b629fedcc945bf8c55bda4ceedc4ae => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:39.182692 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159e41df-e155-4ccb-a9a0-d461f043a07b None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:39.185396 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f175de29-d37d-4bea-8c79-c420f713db33 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:39.224318 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27571d7-308d-4816-ad9b-b97b5eb4b385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:39.225279 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27571d7-308d-4816-ad9b-b97b5eb4b385 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:39.225750 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27571d7-308d-4816-ad9b-b97b5eb4b385 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:39.226202 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27571d7-308d-4816-ad9b-b97b5eb4b385 None admin] PATH_INFO: `/v3/projects/d5b629fedcc945bf8c55bda4ceedc4ae` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:39.227584 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27571d7-308d-4816-ad9b-b97b5eb4b385 None admin] RBAC: Authorizing `identity:get_project(project_id=d5b629fedcc945bf8c55bda4ceedc4ae)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:39.229306 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27571d7-308d-4816-ad9b-b97b5eb4b385 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:39.231294 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 103/205] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 16:53:39 2022] GET /identity/v3/projects/d5b629fedcc945bf8c55bda4ceedc4ae => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:39.232327 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27571d7-308d-4816-ad9b-b97b5eb4b385 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:39.288913 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159e41df-e155-4ccb-a9a0-d461f043a07b None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:39.309784 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:39.330146 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c889c7-db34-4944-b696-8ed5c5f1185f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:39.330943 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c889c7-db34-4944-b696-8ed5c5f1185f None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:39.331467 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c889c7-db34-4944-b696-8ed5c5f1185f None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:39.331816 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c889c7-db34-4944-b696-8ed5c5f1185f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:39.332938 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c889c7-db34-4944-b696-8ed5c5f1185f None admin] RBAC: Adding query filter params (role.id=d49353712e134598b13b44d21b1e791e, scope.project.id=62beed06c75a4725ae7834995533d0a0, user.id=f601719176bc414ab763d3f1fd6662d8) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:53:39.333255 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c889c7-db34-4944-b696-8ed5c5f1185f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:39.334727 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c889c7-db34-4944-b696-8ed5c5f1185f None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:39.342448 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 103/206] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 16:53:39 2022] GET /identity/v3/role_assignments?user.id=f601719176bc414ab763d3f1fd6662d8&scope.project.id=62beed06c75a4725ae7834995533d0a0&role.id=d49353712e134598b13b44d21b1e791e => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:39.348683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c889c7-db34-4944-b696-8ed5c5f1185f None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:39.366344 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27571d7-308d-4816-ad9b-b97b5eb4b385 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:39.394788 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:39.438374 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134bff95-beaa-4dea-a3b9-cc576a964fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:39.439554 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134bff95-beaa-4dea-a3b9-cc576a964fdb None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:39.439833 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134bff95-beaa-4dea-a3b9-cc576a964fdb None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:39.440219 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134bff95-beaa-4dea-a3b9-cc576a964fdb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:39.440968 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134bff95-beaa-4dea-a3b9-cc576a964fdb None admin] RBAC: Adding query filter params (role.id=2c6966cfb10f4d72b0242e9fb5e13b17, scope.project.id=d5b629fedcc945bf8c55bda4ceedc4ae, user.id=1301612ad66a4086a33c4b0b0ab7e53a) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:53:39.441520 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134bff95-beaa-4dea-a3b9-cc576a964fdb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:39.443028 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134bff95-beaa-4dea-a3b9-cc576a964fdb None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:39.449591 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 104/207] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 16:53:39 2022] GET /identity/v3/role_assignments?user.id=1301612ad66a4086a33c4b0b0ab7e53a&scope.project.id=d5b629fedcc945bf8c55bda4ceedc4ae&role.id=2c6966cfb10f4d72b0242e9fb5e13b17 => generated 259 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:39.540766 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c889c7-db34-4944-b696-8ed5c5f1185f None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:39.581122 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0a804b-9c4a-4469-9813-aed72f4fafb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:39.581770 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a804b-9c4a-4469-9813-aed72f4fafb1 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:39.582057 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a804b-9c4a-4469-9813-aed72f4fafb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:39.582339 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a804b-9c4a-4469-9813-aed72f4fafb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:39.583148 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0a804b-9c4a-4469-9813-aed72f4fafb1 None admin] RBAC: Adding query filter params (role.id=f41f6f29d9de4b1985c8c8c14f002cf2, scope.project.id=d5b629fedcc945bf8c55bda4ceedc4ae, user.id=1301612ad66a4086a33c4b0b0ab7e53a) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:53:39.583532 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0a804b-9c4a-4469-9813-aed72f4fafb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:39.585100 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0a804b-9c4a-4469-9813-aed72f4fafb1 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:39.591715 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 104/208] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 16:53:39 2022] GET /identity/v3/role_assignments?user.id=1301612ad66a4086a33c4b0b0ab7e53a&scope.project.id=d5b629fedcc945bf8c55bda4ceedc4ae&role.id=f41f6f29d9de4b1985c8c8c14f002cf2 => generated 259 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:40.448339 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef635aa-af93-450a-8ad0-11396a50eaae None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:40.448845 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef635aa-af93-450a-8ad0-11396a50eaae None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:40.448919 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef635aa-af93-450a-8ad0-11396a50eaae None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:40.450350 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 105/209] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:53:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:53:40.455831 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e228251a-52cb-46a2-8e06-06c1538c3e76 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:40.456338 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e228251a-52cb-46a2-8e06-06c1538c3e76 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:40.456814 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e228251a-52cb-46a2-8e06-06c1538c3e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:40.475747 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-e228251a-52cb-46a2-8e06-06c1538c3e76 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:53:40.477254 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-e228251a-52cb-46a2-8e06-06c1538c3e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:53:40.523084 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:40.546853 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 105/210] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:53:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 16:53:40.606945 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e6c8e-a31b-4d80-a00f-600fe375967c None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:40.606945 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e6c8e-a31b-4d80-a00f-600fe375967c None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:40.606945 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e6c8e-a31b-4d80-a00f-600fe375967c None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:40.608079 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 106/211] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:53:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:53:40.614003 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e45e6a2-abf5-43e9-bf29-b6a16b6f5650 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:40.614539 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e45e6a2-abf5-43e9-bf29-b6a16b6f5650 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:40.615065 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e45e6a2-abf5-43e9-bf29-b6a16b6f5650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:40.634392 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-2e45e6a2-abf5-43e9-bf29-b6a16b6f5650 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:53:40.635946 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-2e45e6a2-abf5-43e9-bf29-b6a16b6f5650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:53:40.706821 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 106/212] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:53:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:53:40.824392 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192e6c8e-a31b-4d80-a00f-600fe375967c None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:40.932733 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192e6c8e-a31b-4d80-a00f-600fe375967c None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:40.967378 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af567ee-7ccb-4074-86aa-576823bbb50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:40.967938 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af567ee-7ccb-4074-86aa-576823bbb50e None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:40.968462 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af567ee-7ccb-4074-86aa-576823bbb50e None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:40.968747 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af567ee-7ccb-4074-86aa-576823bbb50e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:40.980326 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af567ee-7ccb-4074-86aa-576823bbb50e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:53:40.984995 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-1af567ee-7ccb-4074-86aa-576823bbb50e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 16:53:40.984995 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:53:40.984995 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:53:40.984995 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:53:40.984995 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:53:40.984995 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:53:40.984995 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:53:40.984995 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:53:40.984995 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:53:40.984995 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:53:40.984995 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:53:40.984995 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:53:40.984995 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:53:40.984995 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:53:40.984995 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:53:40.984995 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:53:40.986541 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:53:40.986541 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:53:40.986541 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:53:40.986541 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:53:40.986541 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:53:40.986541 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:53:40.986541 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:53:40.986541 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:53:40.986541 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:53:40.986541 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:53:40.986541 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:53:40.986541 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:53:40.986541 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:53:40.986541 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:53:40.986541 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:53:40.986541 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:53:40.987396 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:53:40.987396 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:53:40.987396 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:53:40.987396 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:53:40.987396 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:53:40.987396 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:53:40.987396 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:53:40.987396 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:53:40.987396 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:53:40.987396 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:53:40.987396 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 16:53:40.987396 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:53:40.987396 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 107/213] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:53:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:53:40.994223 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e45e6a2-abf5-43e9-bf29-b6a16b6f5650 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:41.025920 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af567ee-7ccb-4074-86aa-576823bbb50e None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:41.080669 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:41.105803 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e45e6a2-abf5-43e9-bf29-b6a16b6f5650 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:41.117366 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:41.153508 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f81ea0f-6545-4b40-bee8-b0c32aadda5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:41.154374 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f81ea0f-6545-4b40-bee8-b0c32aadda5a None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:41.154663 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f81ea0f-6545-4b40-bee8-b0c32aadda5a None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:41.154973 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f81ea0f-6545-4b40-bee8-b0c32aadda5a None admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:41.155639 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f81ea0f-6545-4b40-bee8-b0c32aadda5a None admin] RBAC: Adding query filter params (name=member) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:53:41.155947 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f81ea0f-6545-4b40-bee8-b0c32aadda5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:41.158807 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f81ea0f-6545-4b40-bee8-b0c32aadda5a None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:41.166644 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af567ee-7ccb-4074-86aa-576823bbb50e None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:41.175414 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 107/214] 10.222.0.82 () {56 vars in 1101 bytes} [Thu Aug 18 16:53:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:41.179930 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f81ea0f-6545-4b40-bee8-b0c32aadda5a None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:41.212409 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63692c31-a2bb-4463-be8a-2ef22fe558c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:41.213324 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63692c31-a2bb-4463-be8a-2ef22fe558c4 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:41.213615 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63692c31-a2bb-4463-be8a-2ef22fe558c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:41.213896 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63692c31-a2bb-4463-be8a-2ef22fe558c4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:41.215363 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63692c31-a2bb-4463-be8a-2ef22fe558c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:41.217322 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63692c31-a2bb-4463-be8a-2ef22fe558c4 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:41.219496 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 108/215] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:53:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:53:41.225718 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63692c31-a2bb-4463-be8a-2ef22fe558c4 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:41.298893 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f81ea0f-6545-4b40-bee8-b0c32aadda5a None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:41.326930 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63692c31-a2bb-4463-be8a-2ef22fe558c4 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:41.329804 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:41.338063 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7237820f-425d-4952-8562-c12b274708e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:41.339017 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237820f-425d-4952-8562-c12b274708e1 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:41.340859 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237820f-425d-4952-8562-c12b274708e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:41.340859 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237820f-425d-4952-8562-c12b274708e1 None admin] PATH_INFO: `/v3/users/8fc79a3f85464a8685c64e77f557f05f` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:41.356073 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7237820f-425d-4952-8562-c12b274708e1 None admin] RBAC: Authorizing `identity:get_user(user_id=8fc79a3f85464a8685c64e77f557f05f)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:41.357967 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7237820f-425d-4952-8562-c12b274708e1 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:41.366491 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 108/216] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 16:53:41 2022] GET /identity/v3/users/8fc79a3f85464a8685c64e77f557f05f => generated 297 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:41.367464 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7237820f-425d-4952-8562-c12b274708e1 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:41.372388 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1891f9-0187-4271-be28-4899d8b98ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:41.372858 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1891f9-0187-4271-be28-4899d8b98ef2 None admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:41.373342 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1891f9-0187-4271-be28-4899d8b98ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:41.373813 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1891f9-0187-4271-be28-4899d8b98ef2 None admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:41.374664 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1891f9-0187-4271-be28-4899d8b98ef2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:41.376918 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1891f9-0187-4271-be28-4899d8b98ef2 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:41.400714 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:41.476340 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7237820f-425d-4952-8562-c12b274708e1 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:41.487160 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 109/217] 10.222.0.82 () {60 vars in 1132 bytes} [Thu Aug 18 16:53:41 2022] POST /identity/v3/users => generated 297 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 16:53:41.508131 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27d2d1a-0ab2-42da-b7f7-8a0df30b7b81 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:41.508514 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27d2d1a-0ab2-42da-b7f7-8a0df30b7b81 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:41.508893 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27d2d1a-0ab2-42da-b7f7-8a0df30b7b81 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:41.510372 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 110/218] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:53:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:53:41.515972 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dd6111-c1d4-4d6e-9c43-77e3a8c32036 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:41.516276 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dd6111-c1d4-4d6e-9c43-77e3a8c32036 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:41.516536 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dd6111-c1d4-4d6e-9c43-77e3a8c32036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:41.536334 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5c73c4-68dd-4971-980b-e826b38dda8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:41.537359 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5c73c4-68dd-4971-980b-e826b38dda8b None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:41.537846 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5c73c4-68dd-4971-980b-e826b38dda8b None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:41.538322 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5c73c4-68dd-4971-980b-e826b38dda8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:41.538577 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-e5dd6111-c1d4-4d6e-9c43-77e3a8c32036 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:53:41.539944 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-e5dd6111-c1d4-4d6e-9c43-77e3a8c32036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:53:41.620776 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 111/219] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:53:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 16:53:41.622402 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a55847-493a-4d72-8ff0-17753615c472 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:41.622692 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a55847-493a-4d72-8ff0-17753615c472 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:41.623000 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a55847-493a-4d72-8ff0-17753615c472 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:41.624461 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 112/220] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:53:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:53:41.630306 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2917387-fb63-4222-bada-d1faa4cafaa4 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:41.630601 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2917387-fb63-4222-bada-d1faa4cafaa4 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:41.630877 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2917387-fb63-4222-bada-d1faa4cafaa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:41.638789 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5c73c4-68dd-4971-980b-e826b38dda8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:41.640354 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5c73c4-68dd-4971-980b-e826b38dda8b None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:41.653444 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-a2917387-fb63-4222-bada-d1faa4cafaa4 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:53:41.654642 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-a2917387-fb63-4222-bada-d1faa4cafaa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:53:41.678883 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:41.751550 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 113/221] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:53:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:53:41.780590 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 109/222] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:53:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 16:53:41.788224 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2917387-fb63-4222-bada-d1faa4cafaa4 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:41.812438 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:41.904467 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2917387-fb63-4222-bada-d1faa4cafaa4 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:41.912531 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5c73c4-68dd-4971-980b-e826b38dda8b None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:41.949270 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8da0570-ab4c-4924-8c13-97ef1f332d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:41.950155 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8da0570-ab4c-4924-8c13-97ef1f332d98 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:41.950460 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8da0570-ab4c-4924-8c13-97ef1f332d98 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:41.950748 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8da0570-ab4c-4924-8c13-97ef1f332d98 None admin] PATH_INFO: `/v3/projects/d5b629fedcc945bf8c55bda4ceedc4ae` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:41.953419 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8da0570-ab4c-4924-8c13-97ef1f332d98 None admin] RBAC: Authorizing `identity:get_project(project_id=d5b629fedcc945bf8c55bda4ceedc4ae)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:41.955792 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8da0570-ab4c-4924-8c13-97ef1f332d98 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:41.958591 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 114/223] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 16:53:41 2022] GET /identity/v3/projects/d5b629fedcc945bf8c55bda4ceedc4ae => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:41.963887 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8da0570-ab4c-4924-8c13-97ef1f332d98 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:42.064012 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:42.073595 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5c73c4-68dd-4971-980b-e826b38dda8b None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:42.077322 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8da0570-ab4c-4924-8c13-97ef1f332d98 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:42.108289 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:42.117782 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581d682c-42ee-49c8-855f-c470d6404655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:42.118531 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581d682c-42ee-49c8-855f-c470d6404655 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:42.118822 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581d682c-42ee-49c8-855f-c470d6404655 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:42.120695 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581d682c-42ee-49c8-855f-c470d6404655 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:42.121504 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee00164-fa2d-4a67-b62a-6b5b29fab6dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:42.122420 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee00164-fa2d-4a67-b62a-6b5b29fab6dc None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:42.122946 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee00164-fa2d-4a67-b62a-6b5b29fab6dc None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:42.123416 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee00164-fa2d-4a67-b62a-6b5b29fab6dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:42.124339 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee00164-fa2d-4a67-b62a-6b5b29fab6dc None admin] RBAC: Adding query filter params (role.id=36d6d323d9e64e659591dbaea3081bcc, scope.project.id=d5b629fedcc945bf8c55bda4ceedc4ae, user.id=8fc79a3f85464a8685c64e77f557f05f) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:53:42.124840 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee00164-fa2d-4a67-b62a-6b5b29fab6dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:42.126555 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee00164-fa2d-4a67-b62a-6b5b29fab6dc None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:42.131512 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581d682c-42ee-49c8-855f-c470d6404655 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:53:42.133270 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 115/224] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 16:53:41 2022] GET /identity/v3/role_assignments?user.id=8fc79a3f85464a8685c64e77f557f05f&scope.project.id=d5b629fedcc945bf8c55bda4ceedc4ae&role.id=36d6d323d9e64e659591dbaea3081bcc => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:42.134278 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee00164-fa2d-4a67-b62a-6b5b29fab6dc None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:42.145600 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-581d682c-42ee-49c8-855f-c470d6404655 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 16:53:42.145600 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:53:42.145600 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:53:42.145600 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:53:42.145600 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:53:42.145600 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:53:42.145600 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:53:42.145600 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:53:42.145600 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:53:42.145600 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:53:42.145600 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:53:42.145600 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:53:42.145600 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:53:42.145600 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:53:42.145600 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:53:42.145600 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:53:42.146685 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:53:42.146685 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:53:42.146685 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:53:42.146685 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:53:42.146685 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:53:42.146685 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:53:42.146685 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:53:42.146685 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:53:42.146685 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:53:42.146685 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:53:42.146685 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:53:42.146685 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:53:42.146685 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:53:42.146685 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:53:42.146685 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:53:42.146685 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:53:42.147680 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:53:42.147680 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:53:42.147680 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:53:42.147680 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:53:42.147680 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:53:42.147680 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:53:42.147680 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:53:42.147680 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:53:42.147680 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:53:42.147680 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:53:42.147680 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 16:53:42.147680 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:53:42.152593 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 110/225] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 16:53:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 16:53:42.158914 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581d682c-42ee-49c8-855f-c470d6404655 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:42.263845 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee00164-fa2d-4a67-b62a-6b5b29fab6dc None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:42.304584 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cfaf56-5789-4061-b94d-b706bd40b680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:42.305592 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cfaf56-5789-4061-b94d-b706bd40b680 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:42.306037 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cfaf56-5789-4061-b94d-b706bd40b680 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:42.306537 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cfaf56-5789-4061-b94d-b706bd40b680 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:42.319572 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cfaf56-5789-4061-b94d-b706bd40b680 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:53:42.325502 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-b8cfaf56-5789-4061-b94d-b706bd40b680 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 16:53:42.325502 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:53:42.325502 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:53:42.325502 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:53:42.325502 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:53:42.325502 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:53:42.325502 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:53:42.325502 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:53:42.325502 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:53:42.325502 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:53:42.325502 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:53:42.325502 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:53:42.325502 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:53:42.325502 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:53:42.325502 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:53:42.325502 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:53:42.326572 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:53:42.326572 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:53:42.326572 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:53:42.326572 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:53:42.326572 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:53:42.326572 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:53:42.326572 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:53:42.326572 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:53:42.326572 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:53:42.326572 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:53:42.326572 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:53:42.326572 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:53:42.326572 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:53:42.326572 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:53:42.326572 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:53:42.326572 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:53:42.327579 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:53:42.327579 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:53:42.327579 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:53:42.327579 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:53:42.327579 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:53:42.327579 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:53:42.327579 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:53:42.327579 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:53:42.327579 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:53:42.327579 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:53:42.327579 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 16:53:42.327579 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:53:42.330053 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 116/226] 10.222.0.82 () {56 vars in 1114 bytes} [Thu Aug 18 16:53:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:53:42.335509 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cfaf56-5789-4061-b94d-b706bd40b680 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:42.354798 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:42.370310 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581d682c-42ee-49c8-855f-c470d6404655 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:42.448179 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a71ad1-fa2e-4eed-b479-af73b8207220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:42.448811 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a71ad1-fa2e-4eed-b479-af73b8207220 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:42.449101 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a71ad1-fa2e-4eed-b479-af73b8207220 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:42.449381 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a71ad1-fa2e-4eed-b479-af73b8207220 None admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:42.450056 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a71ad1-fa2e-4eed-b479-af73b8207220 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:53:42.450367 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a71ad1-fa2e-4eed-b479-af73b8207220 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:42.450468 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cfaf56-5789-4061-b94d-b706bd40b680 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:42.451808 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a71ad1-fa2e-4eed-b479-af73b8207220 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:42.470515 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 111/227] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:53:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:53:42.479951 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a71ad1-fa2e-4eed-b479-af73b8207220 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:42.489695 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff05d1-d942-4a6f-a4c5-b788e734dbd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:42.490952 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff05d1-d942-4a6f-a4c5-b788e734dbd8 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:42.491448 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff05d1-d942-4a6f-a4c5-b788e734dbd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:42.491917 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff05d1-d942-4a6f-a4c5-b788e734dbd8 None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:42.492881 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fff05d1-d942-4a6f-a4c5-b788e734dbd8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:53:42.493376 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fff05d1-d942-4a6f-a4c5-b788e734dbd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:42.494996 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fff05d1-d942-4a6f-a4c5-b788e734dbd8 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:42.504738 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 117/228] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 16:53:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:53:42.514610 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff05d1-d942-4a6f-a4c5-b788e734dbd8 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:42.576431 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:42.655032 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:42.672745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a71ad1-fa2e-4eed-b479-af73b8207220 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:42.701739 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff05d1-d942-4a6f-a4c5-b788e734dbd8 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:42.739284 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40634d3e-578a-490f-a108-37b095f3163b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:42.739930 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40634d3e-578a-490f-a108-37b095f3163b None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:42.740213 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40634d3e-578a-490f-a108-37b095f3163b None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:42.740488 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40634d3e-578a-490f-a108-37b095f3163b None admin] PATH_INFO: `/v3/users/1301612ad66a4086a33c4b0b0ab7e53a` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:42.742722 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40634d3e-578a-490f-a108-37b095f3163b None admin] RBAC: Authorizing `identity:get_user(user_id=1301612ad66a4086a33c4b0b0ab7e53a)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:42.744225 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40634d3e-578a-490f-a108-37b095f3163b None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:42.747474 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 112/229] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 16:53:42 2022] GET /identity/v3/users/1301612ad66a4086a33c4b0b0ab7e53a => generated 283 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:42.753630 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40634d3e-578a-490f-a108-37b095f3163b None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:42.776571 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48fd2d1-62f8-4f2a-8e52-69a89e219757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:42.777328 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48fd2d1-62f8-4f2a-8e52-69a89e219757 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:42.777554 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48fd2d1-62f8-4f2a-8e52-69a89e219757 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:42.777829 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48fd2d1-62f8-4f2a-8e52-69a89e219757 None admin] PATH_INFO: `/v3/users/1301612ad66a4086a33c4b0b0ab7e53a` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:42.779109 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48fd2d1-62f8-4f2a-8e52-69a89e219757 None admin] RBAC: Authorizing `identity:get_user(user_id=1301612ad66a4086a33c4b0b0ab7e53a)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:42.782230 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48fd2d1-62f8-4f2a-8e52-69a89e219757 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:42.782230 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 118/230] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 16:53:42 2022] GET /identity/v3/users/1301612ad66a4086a33c4b0b0ab7e53a => generated 283 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:42.792626 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48fd2d1-62f8-4f2a-8e52-69a89e219757 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:42.946623 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40634d3e-578a-490f-a108-37b095f3163b None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:42.979339 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:43.002914 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48fd2d1-62f8-4f2a-8e52-69a89e219757 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:43.016355 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4129f8f1-595f-42a8-ab9f-9a766ca6cd24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:43.017126 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4129f8f1-595f-42a8-ab9f-9a766ca6cd24 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:43.017407 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4129f8f1-595f-42a8-ab9f-9a766ca6cd24 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:43.017686 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4129f8f1-595f-42a8-ab9f-9a766ca6cd24 None admin] PATH_INFO: `/v3/projects/d5b629fedcc945bf8c55bda4ceedc4ae` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:43.020799 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4129f8f1-595f-42a8-ab9f-9a766ca6cd24 None admin] RBAC: Authorizing `identity:get_project(project_id=d5b629fedcc945bf8c55bda4ceedc4ae)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:43.022290 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4129f8f1-595f-42a8-ab9f-9a766ca6cd24 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:43.024137 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 113/231] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 16:53:42 2022] GET /identity/v3/projects/d5b629fedcc945bf8c55bda4ceedc4ae => generated 302 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:53:43.030430 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4129f8f1-595f-42a8-ab9f-9a766ca6cd24 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:43.080967 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeb06b9-9774-47f8-be49-1d6043dbb27b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:43.081669 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeb06b9-9774-47f8-be49-1d6043dbb27b None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:43.081961 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeb06b9-9774-47f8-be49-1d6043dbb27b None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:43.082236 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeb06b9-9774-47f8-be49-1d6043dbb27b None admin] PATH_INFO: `/v3/projects/d5b629fedcc945bf8c55bda4ceedc4ae` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:43.083519 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeb06b9-9774-47f8-be49-1d6043dbb27b None admin] RBAC: Authorizing `identity:get_project(project_id=d5b629fedcc945bf8c55bda4ceedc4ae)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:43.084967 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeb06b9-9774-47f8-be49-1d6043dbb27b None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:43.096970 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 119/232] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 16:53:42 2022] GET /identity/v3/projects/d5b629fedcc945bf8c55bda4ceedc4ae => generated 302 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:43.108158 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeb06b9-9774-47f8-be49-1d6043dbb27b None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:43.126616 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:43.221624 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4129f8f1-595f-42a8-ab9f-9a766ca6cd24 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:43.259685 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ff6b92-72b2-4b92-b529-deaa2fd363d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:43.260359 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff6b92-72b2-4b92-b529-deaa2fd363d8 None admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:43.260638 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff6b92-72b2-4b92-b529-deaa2fd363d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:43.260912 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff6b92-72b2-4b92-b529-deaa2fd363d8 None admin] PATH_INFO: `/v3/projects/d5b629fedcc945bf8c55bda4ceedc4ae/users/1301612ad66a4086a33c4b0b0ab7e53a/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:43.265184 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ff6b92-72b2-4b92-b529-deaa2fd363d8 None admin] RBAC: Authorizing `identity:create_grant(project_id=d5b629fedcc945bf8c55bda4ceedc4ae, user_id=1301612ad66a4086a33c4b0b0ab7e53a, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:43.266662 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ff6b92-72b2-4b92-b529-deaa2fd363d8 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:43.303516 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeb06b9-9774-47f8-be49-1d6043dbb27b None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:43.342227 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de85c75-6d31-4e82-b7cc-a23fbf493b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:43.342954 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de85c75-6d31-4e82-b7cc-a23fbf493b48 None admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:43.343546 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de85c75-6d31-4e82-b7cc-a23fbf493b48 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:43.343828 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de85c75-6d31-4e82-b7cc-a23fbf493b48 None admin] PATH_INFO: `/v3/projects/d5b629fedcc945bf8c55bda4ceedc4ae/users/1301612ad66a4086a33c4b0b0ab7e53a/roles/f41f6f29d9de4b1985c8c8c14f002cf2` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:43.346785 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de85c75-6d31-4e82-b7cc-a23fbf493b48 None admin] RBAC: Authorizing `identity:create_grant(project_id=d5b629fedcc945bf8c55bda4ceedc4ae, user_id=1301612ad66a4086a33c4b0b0ab7e53a, role_id=f41f6f29d9de4b1985c8c8c14f002cf2)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:43.348414 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de85c75-6d31-4e82-b7cc-a23fbf493b48 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:43.369784 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 114/233] 10.222.0.82 () {58 vars in 1439 bytes} [Thu Aug 18 16:53:43 2022] PUT /identity/v3/projects/d5b629fedcc945bf8c55bda4ceedc4ae/users/1301612ad66a4086a33c4b0b0ab7e53a/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 16:53:43.458372 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 120/234] 10.222.0.82 () {58 vars in 1439 bytes} [Thu Aug 18 16:53:43 2022] PUT /identity/v3/projects/d5b629fedcc945bf8c55bda4ceedc4ae/users/1301612ad66a4086a33c4b0b0ab7e53a/roles/f41f6f29d9de4b1985c8c8c14f002cf2 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 16:53:43.904026 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb55b23-3385-42d7-a4a7-16d7859e57b1 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:43.904530 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb55b23-3385-42d7-a4a7-16d7859e57b1 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:43.904603 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb55b23-3385-42d7-a4a7-16d7859e57b1 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:43.906210 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 115/235] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:53:43 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:53:43.912588 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4491aca6-e005-4eb8-8b1a-0c5089c2de78 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:43.913020 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4491aca6-e005-4eb8-8b1a-0c5089c2de78 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:43.913191 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4491aca6-e005-4eb8-8b1a-0c5089c2de78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:43.931430 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-4491aca6-e005-4eb8-8b1a-0c5089c2de78 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:53:43.932605 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-4491aca6-e005-4eb8-8b1a-0c5089c2de78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:53:43.960729 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bee747-eeb1-4eb8-91cf-51b98b2c3f8b None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:43.961031 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bee747-eeb1-4eb8-91cf-51b98b2c3f8b None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:43.961276 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bee747-eeb1-4eb8-91cf-51b98b2c3f8b None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:43.962615 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 116/236] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:53:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:53:43.980397 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f32a69-b820-4219-8a6f-73af607d77a2 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:43.980692 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f32a69-b820-4219-8a6f-73af607d77a2 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:43.980949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f32a69-b820-4219-8a6f-73af607d77a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:44.002610 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-53f32a69-b820-4219-8a6f-73af607d77a2 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:53:44.003913 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-53f32a69-b820-4219-8a6f-73af607d77a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:53:44.030425 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:44.037561 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 121/237] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:53:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 16:53:44.077226 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:44.106654 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 117/238] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:53:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 16:53:44.359933 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4491aca6-e005-4eb8-8b1a-0c5089c2de78 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:44.462963 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4491aca6-e005-4eb8-8b1a-0c5089c2de78 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:44.483207 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f32a69-b820-4219-8a6f-73af607d77a2 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:44.503150 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e51ffb0-4ab5-4722-99f3-5a21371a3093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:44.503789 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e51ffb0-4ab5-4722-99f3-5a21371a3093 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:44.504065 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e51ffb0-4ab5-4722-99f3-5a21371a3093 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:44.504332 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e51ffb0-4ab5-4722-99f3-5a21371a3093 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:44.521605 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e51ffb0-4ab5-4722-99f3-5a21371a3093 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:53:44.526685 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-4e51ffb0-4ab5-4722-99f3-5a21371a3093 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 16:53:44.526685 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:53:44.526685 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:53:44.526685 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:53:44.526685 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:53:44.526685 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:53:44.526685 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:53:44.526685 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:53:44.526685 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:53:44.526685 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:53:44.526685 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:53:44.526685 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:53:44.526685 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:53:44.526685 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:53:44.526685 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:53:44.526685 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:53:44.527541 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:53:44.527541 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:53:44.527541 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:53:44.527541 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:53:44.527541 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:53:44.527541 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:53:44.527541 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:53:44.527541 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:53:44.527541 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:53:44.527541 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:53:44.527541 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:53:44.527541 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:53:44.527541 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:53:44.527541 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:53:44.527541 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:53:44.527541 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:53:44.528294 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:53:44.528294 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:53:44.528294 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:53:44.528294 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:53:44.528294 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:53:44.528294 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:53:44.528294 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:53:44.528294 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:53:44.528294 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:53:44.528294 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:53:44.528294 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 16:53:44.528294 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:53:44.528294 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 122/239] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:53:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:53:44.534712 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e51ffb0-4ab5-4722-99f3-5a21371a3093 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:44.616492 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:44.627918 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f32a69-b820-4219-8a6f-73af607d77a2 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:44.659904 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e51ffb0-4ab5-4722-99f3-5a21371a3093 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:44.664555 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf59a9f-6129-4944-99a1-9b47f5fe9fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:44.665201 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf59a9f-6129-4944-99a1-9b47f5fe9fd9 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:44.665484 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf59a9f-6129-4944-99a1-9b47f5fe9fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:44.665760 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf59a9f-6129-4944-99a1-9b47f5fe9fd9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:44.666879 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf59a9f-6129-4944-99a1-9b47f5fe9fd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:44.668496 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf59a9f-6129-4944-99a1-9b47f5fe9fd9 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:44.670182 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 118/240] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:53:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:53:44.676578 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac1031-a56e-42d2-b545-73f24d1ab870 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:44.676855 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac1031-a56e-42d2-b545-73f24d1ab870 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:44.677138 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac1031-a56e-42d2-b545-73f24d1ab870 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:44.678872 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 119/241] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:53:44 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:53:44.679642 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ac1031-a56e-42d2-b545-73f24d1ab870 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:44.701062 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cda266d-2ab1-4054-a4c5-f36ffa01185e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:44.701677 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda266d-2ab1-4054-a4c5-f36ffa01185e None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:44.701958 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda266d-2ab1-4054-a4c5-f36ffa01185e None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:44.702231 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda266d-2ab1-4054-a4c5-f36ffa01185e None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:44.703037 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cda266d-2ab1-4054-a4c5-f36ffa01185e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:53:44.703339 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cda266d-2ab1-4054-a4c5-f36ffa01185e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:44.704843 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cda266d-2ab1-4054-a4c5-f36ffa01185e None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:44.718485 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 123/242] 10.222.0.82 () {56 vars in 1101 bytes} [Thu Aug 18 16:53:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:53:44.720646 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1154a2-8d2c-4ca2-bd47-68d24f2c6632 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:44.720933 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1154a2-8d2c-4ca2-bd47-68d24f2c6632 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:44.721207 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1154a2-8d2c-4ca2-bd47-68d24f2c6632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:44.734931 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:44.740677 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-cc1154a2-8d2c-4ca2-bd47-68d24f2c6632 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:53:44.741927 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-cc1154a2-8d2c-4ca2-bd47-68d24f2c6632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:53:44.798378 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ac1031-a56e-42d2-b545-73f24d1ab870 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:44.815893 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 124/243] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:53:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:53:44.822058 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1154a2-8d2c-4ca2-bd47-68d24f2c6632 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:44.837765 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce0ea3-878c-4f84-a127-ec70f808185d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:44.838388 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce0ea3-878c-4f84-a127-ec70f808185d None admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:44.838670 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce0ea3-878c-4f84-a127-ec70f808185d None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:44.838970 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce0ea3-878c-4f84-a127-ec70f808185d None admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:44.839733 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ce0ea3-878c-4f84-a127-ec70f808185d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:44.841371 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ce0ea3-878c-4f84-a127-ec70f808185d None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:44.905757 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:44.954770 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 120/244] 10.222.0.82 () {60 vars in 1132 bytes} [Thu Aug 18 16:53:44 2022] POST /identity/v3/users => generated 293 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 16:53:44.957207 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1154a2-8d2c-4ca2-bd47-68d24f2c6632 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:44.998581 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e87daa-6642-4fd4-ac41-241eaf302475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:44.999300 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e87daa-6642-4fd4-ac41-241eaf302475 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:44.999586 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e87daa-6642-4fd4-ac41-241eaf302475 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:44.999861 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e87daa-6642-4fd4-ac41-241eaf302475 None admin] PATH_INFO: `/v3/users/a19066a0433a4376bafbbe454b4f7079` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:45.015194 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e87daa-6642-4fd4-ac41-241eaf302475 None admin] RBAC: Authorizing `identity:get_user(user_id=a19066a0433a4376bafbbe454b4f7079)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:45.017202 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e87daa-6642-4fd4-ac41-241eaf302475 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:45.019449 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 125/245] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 16:53:44 2022] GET /identity/v3/users/a19066a0433a4376bafbbe454b4f7079 => generated 297 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:45.024850 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce0ea3-878c-4f84-a127-ec70f808185d None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:45.116721 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e87daa-6642-4fd4-ac41-241eaf302475 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:45.140416 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce0ea3-878c-4f84-a127-ec70f808185d None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:45.170829 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:45.191749 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b334482-59e1-4e65-98f3-badcd11c6e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:45.192385 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b334482-59e1-4e65-98f3-badcd11c6e7f None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:45.204027 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b334482-59e1-4e65-98f3-badcd11c6e7f None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:45.204027 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b334482-59e1-4e65-98f3-badcd11c6e7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:45.232932 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e87daa-6642-4fd4-ac41-241eaf302475 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:45.259302 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:45.275334 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b6090-9aa3-4504-bcf1-6a1177313e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:45.276399 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b6090-9aa3-4504-bcf1-6a1177313e0e None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:45.276818 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b6090-9aa3-4504-bcf1-6a1177313e0e None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:45.277224 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b6090-9aa3-4504-bcf1-6a1177313e0e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:45.284735 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b334482-59e1-4e65-98f3-badcd11c6e7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:45.286838 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b334482-59e1-4e65-98f3-badcd11c6e7f None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:45.291050 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331b6090-9aa3-4504-bcf1-6a1177313e0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:53:45.296200 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-331b6090-9aa3-4504-bcf1-6a1177313e0e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 16:53:45.296200 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:53:45.296200 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:53:45.296200 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:53:45.296200 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:53:45.296200 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:53:45.296200 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:53:45.296200 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:53:45.296200 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:53:45.296200 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:53:45.296200 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:53:45.296200 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:53:45.296200 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:53:45.296200 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:53:45.296200 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:53:45.296200 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:53:45.297066 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:53:45.297066 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:53:45.297066 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:53:45.297066 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:53:45.297066 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:53:45.297066 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:53:45.297066 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:53:45.297066 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:53:45.297066 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:53:45.297066 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:53:45.297066 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:53:45.297066 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:53:45.297066 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:53:45.297066 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:53:45.297066 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:53:45.297066 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:53:45.298355 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:53:45.298355 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:53:45.298355 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:53:45.298355 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:53:45.298355 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:53:45.298355 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:53:45.298355 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:53:45.298355 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:53:45.298355 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:53:45.298355 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:53:45.298355 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 16:53:45.298355 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:53:45.303314 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 126/246] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:53:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:53:45.307345 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b6090-9aa3-4504-bcf1-6a1177313e0e None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:45.415591 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:45.423041 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 121/247] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:53:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 18 16:53:45.431839 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b334482-59e1-4e65-98f3-badcd11c6e7f None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:45.432501 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b6090-9aa3-4504-bcf1-6a1177313e0e None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:45.477168 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dfa634-3071-4571-80ae-e3dae4b41ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:45.477380 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dfa634-3071-4571-80ae-e3dae4b41ddf None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:45.477739 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dfa634-3071-4571-80ae-e3dae4b41ddf None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:45.478106 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dfa634-3071-4571-80ae-e3dae4b41ddf None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:45.478884 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dfa634-3071-4571-80ae-e3dae4b41ddf None admin] RBAC: Adding query filter params (name=member) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:53:45.479310 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dfa634-3071-4571-80ae-e3dae4b41ddf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:45.480777 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dfa634-3071-4571-80ae-e3dae4b41ddf None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:45.490718 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 127/248] 10.222.0.82 () {56 vars in 1101 bytes} [Thu Aug 18 16:53:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:53:45.496282 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dfa634-3071-4571-80ae-e3dae4b41ddf None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:45.531600 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:45.552603 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b334482-59e1-4e65-98f3-badcd11c6e7f None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:45.591755 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a29cf3c-ccba-4ad4-95be-c9fbead9bbc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:45.592774 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29cf3c-ccba-4ad4-95be-c9fbead9bbc0 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:45.593516 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29cf3c-ccba-4ad4-95be-c9fbead9bbc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:45.593963 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29cf3c-ccba-4ad4-95be-c9fbead9bbc0 None admin] PATH_INFO: `/v3/projects/d5b629fedcc945bf8c55bda4ceedc4ae` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:45.595707 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a29cf3c-ccba-4ad4-95be-c9fbead9bbc0 None admin] RBAC: Authorizing `identity:get_project(project_id=d5b629fedcc945bf8c55bda4ceedc4ae)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:45.597487 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a29cf3c-ccba-4ad4-95be-c9fbead9bbc0 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:45.599777 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 122/249] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 16:53:45 2022] GET /identity/v3/projects/d5b629fedcc945bf8c55bda4ceedc4ae => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:53:45.606594 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a29cf3c-ccba-4ad4-95be-c9fbead9bbc0 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:45.614665 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dfa634-3071-4571-80ae-e3dae4b41ddf None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:45.660887 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919ca1c4-03b3-4c00-aee5-3989acd6afd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:45.661821 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ca1c4-03b3-4c00-aee5-3989acd6afd9 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:45.662279 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ca1c4-03b3-4c00-aee5-3989acd6afd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:45.663134 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ca1c4-03b3-4c00-aee5-3989acd6afd9 None admin] PATH_INFO: `/v3/users/8fc79a3f85464a8685c64e77f557f05f` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:45.665211 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919ca1c4-03b3-4c00-aee5-3989acd6afd9 None admin] RBAC: Authorizing `identity:get_user(user_id=8fc79a3f85464a8685c64e77f557f05f)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:45.667370 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919ca1c4-03b3-4c00-aee5-3989acd6afd9 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:45.669807 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 128/250] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 16:53:45 2022] GET /identity/v3/users/8fc79a3f85464a8685c64e77f557f05f => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:45.676585 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919ca1c4-03b3-4c00-aee5-3989acd6afd9 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:45.695118 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:45.734814 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a29cf3c-ccba-4ad4-95be-c9fbead9bbc0 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:45.786487 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a62935b-8ede-4ef8-b3de-1e328fac9d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:45.787176 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a62935b-8ede-4ef8-b3de-1e328fac9d41 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:45.787488 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a62935b-8ede-4ef8-b3de-1e328fac9d41 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:45.787746 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a62935b-8ede-4ef8-b3de-1e328fac9d41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:45.788480 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a62935b-8ede-4ef8-b3de-1e328fac9d41 None admin] RBAC: Adding query filter params (role.id=d49353712e134598b13b44d21b1e791e, scope.project.id=d5b629fedcc945bf8c55bda4ceedc4ae, user.id=a19066a0433a4376bafbbe454b4f7079) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:53:45.788951 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a62935b-8ede-4ef8-b3de-1e328fac9d41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:45.790532 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a62935b-8ede-4ef8-b3de-1e328fac9d41 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:45.798269 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:45.802609 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919ca1c4-03b3-4c00-aee5-3989acd6afd9 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:45.803067 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 123/251] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 16:53:45 2022] GET /identity/v3/role_assignments?user.id=a19066a0433a4376bafbbe454b4f7079&scope.project.id=d5b629fedcc945bf8c55bda4ceedc4ae&role.id=d49353712e134598b13b44d21b1e791e => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:53:45.804546 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7bdd5-e0e5-4f7c-a2af-11add317bc87 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:45.804854 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7bdd5-e0e5-4f7c-a2af-11add317bc87 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:45.805152 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7bdd5-e0e5-4f7c-a2af-11add317bc87 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:45.806878 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 124/252] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:53:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:53:45.818324 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0e0c6-fcf9-48b5-b0b8-0d2abb5191a7 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:45.818610 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0e0c6-fcf9-48b5-b0b8-0d2abb5191a7 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:45.818879 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0e0c6-fcf9-48b5-b0b8-0d2abb5191a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:45.851543 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1b56a6-ceb2-42cd-963b-bd99f8d42ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:45.852619 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b56a6-ceb2-42cd-963b-bd99f8d42ded None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:45.853118 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b56a6-ceb2-42cd-963b-bd99f8d42ded None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:45.853586 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b56a6-ceb2-42cd-963b-bd99f8d42ded None admin] PATH_INFO: `/v3/projects/d5b629fedcc945bf8c55bda4ceedc4ae` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:45.855055 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1b56a6-ceb2-42cd-963b-bd99f8d42ded None admin] RBAC: Authorizing `identity:get_project(project_id=d5b629fedcc945bf8c55bda4ceedc4ae)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:45.855352 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-12c0e0c6-fcf9-48b5-b0b8-0d2abb5191a7 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:53:45.856562 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-12c0e0c6-fcf9-48b5-b0b8-0d2abb5191a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:53:45.857155 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1b56a6-ceb2-42cd-963b-bd99f8d42ded None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:45.859803 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 129/253] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 16:53:45 2022] GET /identity/v3/projects/d5b629fedcc945bf8c55bda4ceedc4ae => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:53:45.871400 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1b56a6-ceb2-42cd-963b-bd99f8d42ded None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:45.954423 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 125/254] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:53:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:53:45.960529 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21eda7-e36c-44ba-88eb-1df2046c0f9b None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:45.961128 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21eda7-e36c-44ba-88eb-1df2046c0f9b None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:45.961128 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21eda7-e36c-44ba-88eb-1df2046c0f9b None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:45.967271 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 126/255] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:53:45 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:53:45.968896 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fae2f7-9ab1-4ad2-812d-534f9fb5f58d None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:45.969265 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fae2f7-9ab1-4ad2-812d-534f9fb5f58d None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:45.970108 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fae2f7-9ab1-4ad2-812d-534f9fb5f58d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:46.007602 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1b56a6-ceb2-42cd-963b-bd99f8d42ded None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:46.024797 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-d4fae2f7-9ab1-4ad2-812d-534f9fb5f58d None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:53:46.026015 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-d4fae2f7-9ab1-4ad2-812d-534f9fb5f58d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:53:46.055407 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00bc790-7173-495d-b479-56deb793af4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:46.056424 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00bc790-7173-495d-b479-56deb793af4a None admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:46.056720 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00bc790-7173-495d-b479-56deb793af4a None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:46.056993 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00bc790-7173-495d-b479-56deb793af4a None admin] PATH_INFO: `/v3/projects/d5b629fedcc945bf8c55bda4ceedc4ae/users/8fc79a3f85464a8685c64e77f557f05f/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:46.060452 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00bc790-7173-495d-b479-56deb793af4a None admin] RBAC: Authorizing `identity:create_grant(project_id=d5b629fedcc945bf8c55bda4ceedc4ae, user_id=8fc79a3f85464a8685c64e77f557f05f, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:46.063089 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00bc790-7173-495d-b479-56deb793af4a None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:46.099318 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:46.120966 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 127/256] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:53:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:53:46.154963 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 130/257] 10.222.0.82 () {58 vars in 1439 bytes} [Thu Aug 18 16:53:45 2022] PUT /identity/v3/projects/d5b629fedcc945bf8c55bda4ceedc4ae/users/8fc79a3f85464a8685c64e77f557f05f/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 16:53:46.285486 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fae2f7-9ab1-4ad2-812d-534f9fb5f58d None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:46.407829 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fae2f7-9ab1-4ad2-812d-534f9fb5f58d None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:46.450443 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc670a90-c739-400c-a0aa-9ad0db9cbb84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:46.451121 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc670a90-c739-400c-a0aa-9ad0db9cbb84 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:46.451408 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc670a90-c739-400c-a0aa-9ad0db9cbb84 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:46.451685 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc670a90-c739-400c-a0aa-9ad0db9cbb84 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:46.462703 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc670a90-c739-400c-a0aa-9ad0db9cbb84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:53:46.471661 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-dc670a90-c739-400c-a0aa-9ad0db9cbb84 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 16:53:46.471661 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:53:46.471661 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:53:46.471661 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:53:46.471661 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:53:46.471661 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:53:46.471661 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:53:46.471661 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:53:46.471661 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:53:46.471661 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:53:46.471661 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:53:46.471661 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:53:46.471661 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:53:46.471661 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:53:46.471661 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:53:46.471661 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:53:46.472518 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:53:46.472518 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:53:46.472518 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:53:46.472518 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:53:46.472518 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:53:46.472518 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:53:46.472518 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:53:46.472518 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:53:46.472518 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:53:46.472518 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:53:46.472518 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:53:46.472518 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:53:46.472518 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:53:46.472518 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:53:46.472518 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:53:46.472518 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:53:46.473269 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:53:46.473269 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:53:46.473269 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:53:46.473269 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:53:46.473269 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:53:46.473269 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:53:46.473269 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:53:46.473269 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:53:46.473269 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:53:46.473269 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:53:46.473269 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 16:53:46.473269 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:53:46.473269 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 128/258] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 16:53:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:53:46.480667 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00bc790-7173-495d-b479-56deb793af4a None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:46.534145 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:46.637689 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00bc790-7173-495d-b479-56deb793af4a None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:46.640812 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc670a90-c739-400c-a0aa-9ad0db9cbb84 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:46.710540 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a19742b-3327-4d00-9844-12d542de266e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:46.711183 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19742b-3327-4d00-9844-12d542de266e None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:46.711465 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19742b-3327-4d00-9844-12d542de266e None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:46.711737 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19742b-3327-4d00-9844-12d542de266e None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:46.712405 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a19742b-3327-4d00-9844-12d542de266e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:53:46.712706 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a19742b-3327-4d00-9844-12d542de266e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:46.714087 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a19742b-3327-4d00-9844-12d542de266e None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:46.745002 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 131/259] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:53:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:46.752420 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a19742b-3327-4d00-9844-12d542de266e None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:46.771593 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:46.869212 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc670a90-c739-400c-a0aa-9ad0db9cbb84 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:46.941646 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff11cb0-1725-40da-8c00-a0ee0fde9c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:46.942231 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff11cb0-1725-40da-8c00-a0ee0fde9c86 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:46.942516 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff11cb0-1725-40da-8c00-a0ee0fde9c86 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:46.942791 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff11cb0-1725-40da-8c00-a0ee0fde9c86 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:46.962980 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:46.969774 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff11cb0-1725-40da-8c00-a0ee0fde9c86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:53:46.980895 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-5ff11cb0-1725-40da-8c00-a0ee0fde9c86 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 16:53:46.980895 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:53:46.980895 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:53:46.980895 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:53:46.980895 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:53:46.980895 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:53:46.980895 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:53:46.980895 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:53:46.980895 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:53:46.980895 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:53:46.980895 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:53:46.980895 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:53:46.980895 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:53:46.980895 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:53:46.980895 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:53:46.980895 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:53:46.981789 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:53:46.981789 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:53:46.981789 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:53:46.981789 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:53:46.981789 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:53:46.981789 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:53:46.981789 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:53:46.981789 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:53:46.981789 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:53:46.981789 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:53:46.981789 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:53:46.981789 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:53:46.981789 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:53:46.981789 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:53:46.981789 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:53:46.981789 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:53:46.982530 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:53:46.982530 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:53:46.982530 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:53:46.982530 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:53:46.982530 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:53:46.982530 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:53:46.982530 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:53:46.982530 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:53:46.982530 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:53:46.982530 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:53:46.982530 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 16:53:46.982530 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:53:46.982530 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 129/260] 10.222.0.82 () {56 vars in 1114 bytes} [Thu Aug 18 16:53:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 16:53:46.991144 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a19742b-3327-4d00-9844-12d542de266e None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:46.992576 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff11cb0-1725-40da-8c00-a0ee0fde9c86 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:47.079948 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec3da39-d1f8-4bd5-b537-c6ffb726a256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:47.080732 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3da39-d1f8-4bd5-b537-c6ffb726a256 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:47.081015 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3da39-d1f8-4bd5-b537-c6ffb726a256 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:47.081292 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3da39-d1f8-4bd5-b537-c6ffb726a256 None admin] PATH_INFO: `/v3/users/1301612ad66a4086a33c4b0b0ab7e53a` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:47.087563 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec3da39-d1f8-4bd5-b537-c6ffb726a256 None admin] RBAC: Authorizing `identity:get_user(user_id=1301612ad66a4086a33c4b0b0ab7e53a)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:47.089112 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec3da39-d1f8-4bd5-b537-c6ffb726a256 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:47.095089 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 132/261] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 16:53:46 2022] GET /identity/v3/users/1301612ad66a4086a33c4b0b0ab7e53a => generated 283 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:53:47.114411 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec3da39-d1f8-4bd5-b537-c6ffb726a256 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:47.125049 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff11cb0-1725-40da-8c00-a0ee0fde9c86 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:47.139295 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:47.162402 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8f4892-4693-4c75-82ec-d345dcf7849e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:47.163336 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8f4892-4693-4c75-82ec-d345dcf7849e None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:47.163818 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8f4892-4693-4c75-82ec-d345dcf7849e None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:47.164308 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8f4892-4693-4c75-82ec-d345dcf7849e None admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:47.165160 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8f4892-4693-4c75-82ec-d345dcf7849e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:53:47.165654 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8f4892-4693-4c75-82ec-d345dcf7849e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:47.167253 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8f4892-4693-4c75-82ec-d345dcf7849e None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:47.176408 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 130/262] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 16:53:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:47.181716 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8f4892-4693-4c75-82ec-d345dcf7849e None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:47.291543 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8f4892-4693-4c75-82ec-d345dcf7849e None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:47.292570 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec3da39-d1f8-4bd5-b537-c6ffb726a256 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:47.328610 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3a118d-b053-4028-89fe-3a86a31ec640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:47.329902 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3a118d-b053-4028-89fe-3a86a31ec640 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:47.330412 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3a118d-b053-4028-89fe-3a86a31ec640 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:47.330882 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3a118d-b053-4028-89fe-3a86a31ec640 None admin] PATH_INFO: `/v3/users/1301612ad66a4086a33c4b0b0ab7e53a` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:47.332258 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3a118d-b053-4028-89fe-3a86a31ec640 None admin] RBAC: Authorizing `identity:get_user(user_id=1301612ad66a4086a33c4b0b0ab7e53a)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:47.333959 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3a118d-b053-4028-89fe-3a86a31ec640 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:47.336061 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 131/263] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 16:53:47 2022] GET /identity/v3/users/1301612ad66a4086a33c4b0b0ab7e53a => generated 283 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:53:47.341733 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3a118d-b053-4028-89fe-3a86a31ec640 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:47.362965 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33e52bf-582c-456c-b098-cec371a92ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:47.363953 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33e52bf-582c-456c-b098-cec371a92ed5 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:47.364531 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33e52bf-582c-456c-b098-cec371a92ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:47.364993 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33e52bf-582c-456c-b098-cec371a92ed5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:47.411860 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:47.412645 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:47.475257 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3a118d-b053-4028-89fe-3a86a31ec640 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:47.494444 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33e52bf-582c-456c-b098-cec371a92ed5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:47.495957 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33e52bf-582c-456c-b098-cec371a92ed5 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:47.515962 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475f162a-aa67-439d-ac75-91d09f7cad2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:47.517831 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475f162a-aa67-439d-ac75-91d09f7cad2c None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:47.518476 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475f162a-aa67-439d-ac75-91d09f7cad2c None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:47.520119 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475f162a-aa67-439d-ac75-91d09f7cad2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:47.644990 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475f162a-aa67-439d-ac75-91d09f7cad2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:47.646802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475f162a-aa67-439d-ac75-91d09f7cad2c None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:47.686926 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 133/264] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:53:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 569 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 16:53:47.696225 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:47.704126 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33e52bf-582c-456c-b098-cec371a92ed5 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:47.757807 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 132/265] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:53:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 16:53:47.768498 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475f162a-aa67-439d-ac75-91d09f7cad2c None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:47.839291 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:47.889382 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475f162a-aa67-439d-ac75-91d09f7cad2c None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:47.909656 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33e52bf-582c-456c-b098-cec371a92ed5 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:47.960564 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0aea93-c5e6-4464-8297-e97e94db83ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:47.961213 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0aea93-c5e6-4464-8297-e97e94db83ad None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:47.961492 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0aea93-c5e6-4464-8297-e97e94db83ad None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:47.961764 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0aea93-c5e6-4464-8297-e97e94db83ad None admin] PATH_INFO: `/v3/projects/d5b629fedcc945bf8c55bda4ceedc4ae` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:47.963484 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0aea93-c5e6-4464-8297-e97e94db83ad None admin] RBAC: Authorizing `identity:get_project(project_id=d5b629fedcc945bf8c55bda4ceedc4ae)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:47.964964 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0aea93-c5e6-4464-8297-e97e94db83ad None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:47.966595 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 133/266] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 16:53:47 2022] GET /identity/v3/projects/d5b629fedcc945bf8c55bda4ceedc4ae => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:47.978101 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1366190c-999e-42a8-80af-6fe102ece3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:47.978743 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366190c-999e-42a8-80af-6fe102ece3c7 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:47.979051 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366190c-999e-42a8-80af-6fe102ece3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:47.979326 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366190c-999e-42a8-80af-6fe102ece3c7 None admin] PATH_INFO: `/v3/projects/d5b629fedcc945bf8c55bda4ceedc4ae` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:47.982261 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0aea93-c5e6-4464-8297-e97e94db83ad None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:47.989726 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1366190c-999e-42a8-80af-6fe102ece3c7 None admin] RBAC: Authorizing `identity:get_project(project_id=d5b629fedcc945bf8c55bda4ceedc4ae)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:47.991293 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1366190c-999e-42a8-80af-6fe102ece3c7 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:47.993239 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 134/267] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 16:53:47 2022] GET /identity/v3/projects/d5b629fedcc945bf8c55bda4ceedc4ae => generated 302 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:48.003986 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:48.012911 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1366190c-999e-42a8-80af-6fe102ece3c7 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:48.134876 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1366190c-999e-42a8-80af-6fe102ece3c7 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:48.172412 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a19bb1-ebe2-4c27-bf45-ef54a472312d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:48.173361 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a19bb1-ebe2-4c27-bf45-ef54a472312d None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:48.173654 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a19bb1-ebe2-4c27-bf45-ef54a472312d None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:48.173932 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a19bb1-ebe2-4c27-bf45-ef54a472312d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:48.174664 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a19bb1-ebe2-4c27-bf45-ef54a472312d None admin] RBAC: Adding query filter params (role.id=2c6966cfb10f4d72b0242e9fb5e13b17, scope.project.id=d5b629fedcc945bf8c55bda4ceedc4ae, user.id=1301612ad66a4086a33c4b0b0ab7e53a) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:53:48.174999 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a19bb1-ebe2-4c27-bf45-ef54a472312d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:48.175683 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:48.179582 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a19bb1-ebe2-4c27-bf45-ef54a472312d None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:48.190580 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 135/268] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 16:53:48 2022] GET /identity/v3/role_assignments?user.id=1301612ad66a4086a33c4b0b0ab7e53a&scope.project.id=d5b629fedcc945bf8c55bda4ceedc4ae&role.id=2c6966cfb10f4d72b0242e9fb5e13b17 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:48.229410 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0aea93-c5e6-4464-8297-e97e94db83ad None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:48.280611 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba6ec3f-b83e-4bb2-a6d3-e51872be55ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:48.281238 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba6ec3f-b83e-4bb2-a6d3-e51872be55ff None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:48.281565 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba6ec3f-b83e-4bb2-a6d3-e51872be55ff None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:48.282453 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba6ec3f-b83e-4bb2-a6d3-e51872be55ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:48.283442 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba6ec3f-b83e-4bb2-a6d3-e51872be55ff None admin] RBAC: Adding query filter params (role.id=f41f6f29d9de4b1985c8c8c14f002cf2, scope.project.id=d5b629fedcc945bf8c55bda4ceedc4ae, user.id=1301612ad66a4086a33c4b0b0ab7e53a) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:53:48.283752 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba6ec3f-b83e-4bb2-a6d3-e51872be55ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:48.285215 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba6ec3f-b83e-4bb2-a6d3-e51872be55ff None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:48.292013 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 134/269] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 16:53:47 2022] GET /identity/v3/role_assignments?user.id=1301612ad66a4086a33c4b0b0ab7e53a&scope.project.id=d5b629fedcc945bf8c55bda4ceedc4ae&role.id=f41f6f29d9de4b1985c8c8c14f002cf2 => generated 608 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:53:48.522836 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b9a42-b5a5-4a15-9f23-abb8c4395908 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:48.523043 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b9a42-b5a5-4a15-9f23-abb8c4395908 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:48.523325 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b9a42-b5a5-4a15-9f23-abb8c4395908 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:48.524875 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 136/270] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:53:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:53:48.530105 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa7b9b-4037-4190-b868-c502adfffc91 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:48.530397 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa7b9b-4037-4190-b868-c502adfffc91 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:48.530678 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa7b9b-4037-4190-b868-c502adfffc91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:48.560117 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-91fa7b9b-4037-4190-b868-c502adfffc91 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:53:48.561349 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-91fa7b9b-4037-4190-b868-c502adfffc91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:53:48.618542 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa407792-9ef5-4f66-9fac-1327b3df351f None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:48.618842 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa407792-9ef5-4f66-9fac-1327b3df351f None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:48.619118 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa407792-9ef5-4f66-9fac-1327b3df351f None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:48.620721 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 137/271] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:53:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:53:48.625458 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83809fd-e2b9-46e6-9325-cd7f9332fcc4 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:48.626397 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83809fd-e2b9-46e6-9325-cd7f9332fcc4 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:48.627139 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83809fd-e2b9-46e6-9325-cd7f9332fcc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:48.648189 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 135/272] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:53:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:53:48.654804 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-c83809fd-e2b9-46e6-9325-cd7f9332fcc4 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:53:48.656024 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-c83809fd-e2b9-46e6-9325-cd7f9332fcc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:53:48.693718 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec88ca3-8a94-4474-881b-73880373ba9d None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:48.694021 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec88ca3-8a94-4474-881b-73880373ba9d None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:48.694286 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec88ca3-8a94-4474-881b-73880373ba9d None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:48.695666 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 136/273] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:53:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 16:53:48.701364 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4479b3-4416-447c-a1a8-9f678286de2a None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:48.701647 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4479b3-4416-447c-a1a8-9f678286de2a None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:48.701923 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4479b3-4416-447c-a1a8-9f678286de2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:48.722837 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:48.735607 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 138/274] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:53:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 16:53:48.739542 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-4c4479b3-4416-447c-a1a8-9f678286de2a None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:53:48.740733 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-4c4479b3-4416-447c-a1a8-9f678286de2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:53:48.833446 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 137/275] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:53:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 16:53:48.976632 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83809fd-e2b9-46e6-9325-cd7f9332fcc4 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:49.021199 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4479b3-4416-447c-a1a8-9f678286de2a None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:49.082181 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83809fd-e2b9-46e6-9325-cd7f9332fcc4 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:49.104704 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:49.125238 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82491243-8ba0-4732-ac99-f86bfc23037d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:49.125948 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82491243-8ba0-4732-ac99-f86bfc23037d None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:49.132268 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82491243-8ba0-4732-ac99-f86bfc23037d None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:49.132268 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82491243-8ba0-4732-ac99-f86bfc23037d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:49.132268 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82491243-8ba0-4732-ac99-f86bfc23037d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:49.132268 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82491243-8ba0-4732-ac99-f86bfc23037d None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:49.132268 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 139/276] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:53:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:49.132268 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82491243-8ba0-4732-ac99-f86bfc23037d None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:49.140489 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4479b3-4416-447c-a1a8-9f678286de2a None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:49.175329 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef23e90-65a1-4693-bcc2-d498189005b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:49.175967 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef23e90-65a1-4693-bcc2-d498189005b9 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:49.176250 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef23e90-65a1-4693-bcc2-d498189005b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:49.176532 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef23e90-65a1-4693-bcc2-d498189005b9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:49.178393 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:49.186861 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef23e90-65a1-4693-bcc2-d498189005b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:53:49.191678 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-3ef23e90-65a1-4693-bcc2-d498189005b9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 16:53:49.191678 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:53:49.191678 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:53:49.191678 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:53:49.191678 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:53:49.191678 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:53:49.191678 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:53:49.191678 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:53:49.191678 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:53:49.191678 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:53:49.191678 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:53:49.191678 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:53:49.191678 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:53:49.191678 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:53:49.191678 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:53:49.191678 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:53:49.192561 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:53:49.192561 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:53:49.192561 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:53:49.192561 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:53:49.192561 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:53:49.192561 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:53:49.192561 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:53:49.192561 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:53:49.192561 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:53:49.192561 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:53:49.192561 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:53:49.192561 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:53:49.192561 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:53:49.192561 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:53:49.192561 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:53:49.192561 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:53:49.193303 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:53:49.193303 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:53:49.193303 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:53:49.193303 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:53:49.193303 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:53:49.193303 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:53:49.193303 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:53:49.193303 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:53:49.193303 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:53:49.193303 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:53:49.193303 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 16:53:49.193303 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:53:49.193303 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 138/277] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:53:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:53:49.193927 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef23e90-65a1-4693-bcc2-d498189005b9 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:49.236634 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82491243-8ba0-4732-ac99-f86bfc23037d None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:49.269794 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49db3d64-9e85-4476-bf28-109025b28200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:49.270422 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49db3d64-9e85-4476-bf28-109025b28200 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:49.270703 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49db3d64-9e85-4476-bf28-109025b28200 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:49.271012 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49db3d64-9e85-4476-bf28-109025b28200 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:49.280823 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49db3d64-9e85-4476-bf28-109025b28200 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:53:49.285984 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-49db3d64-9e85-4476-bf28-109025b28200 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 16:53:49.285984 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:53:49.285984 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:53:49.285984 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:53:49.285984 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:53:49.285984 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:53:49.285984 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:53:49.285984 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:53:49.285984 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:53:49.285984 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:53:49.285984 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:53:49.285984 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:53:49.285984 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:53:49.285984 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:53:49.285984 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:53:49.285984 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:53:49.286831 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:53:49.286831 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:53:49.286831 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:53:49.286831 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:53:49.286831 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:53:49.286831 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:53:49.286831 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:53:49.286831 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:53:49.286831 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:53:49.286831 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:53:49.286831 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:53:49.286831 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:53:49.286831 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:53:49.286831 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:53:49.286831 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:53:49.286831 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:53:49.287681 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:53:49.287681 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:53:49.287681 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:53:49.287681 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:53:49.287681 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:53:49.287681 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:53:49.287681 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:53:49.287681 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:53:49.287681 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:53:49.287681 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:53:49.287681 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 16:53:49.287681 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:53:49.287681 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 140/278] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:53:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:53:49.288332 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efec048-e315-4cd5-a3cc-06496d9eb45a None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:49.288508 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efec048-e315-4cd5-a3cc-06496d9eb45a None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:49.288994 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efec048-e315-4cd5-a3cc-06496d9eb45a None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:49.293863 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 141/279] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:53:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:53:49.293863 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efec048-e315-4cd5-a3cc-06496d9eb45a None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:49.296223 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef23e90-65a1-4693-bcc2-d498189005b9 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:49.330811 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c782bdd9-8f15-4cb1-bd07-f378debf40b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:49.331443 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c782bdd9-8f15-4cb1-bd07-f378debf40b6 None admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:49.331730 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c782bdd9-8f15-4cb1-bd07-f378debf40b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:49.334449 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c782bdd9-8f15-4cb1-bd07-f378debf40b6 None admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:49.334449 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c782bdd9-8f15-4cb1-bd07-f378debf40b6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:49.334449 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c782bdd9-8f15-4cb1-bd07-f378debf40b6 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:49.344175 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:49.394106 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efec048-e315-4cd5-a3cc-06496d9eb45a None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:49.426409 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107dad32-bb82-44aa-8fcf-5694830eeb0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:49.427067 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107dad32-bb82-44aa-8fcf-5694830eeb0b None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:49.427353 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107dad32-bb82-44aa-8fcf-5694830eeb0b None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:49.427629 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107dad32-bb82-44aa-8fcf-5694830eeb0b None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:49.428291 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107dad32-bb82-44aa-8fcf-5694830eeb0b None admin] RBAC: Adding query filter params (name=member) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:53:49.428581 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107dad32-bb82-44aa-8fcf-5694830eeb0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:49.429947 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107dad32-bb82-44aa-8fcf-5694830eeb0b None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:49.437554 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 142/280] 10.222.0.82 () {56 vars in 1101 bytes} [Thu Aug 18 16:53:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:49.438524 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107dad32-bb82-44aa-8fcf-5694830eeb0b None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:49.474275 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 139/281] 10.222.0.82 () {60 vars in 1132 bytes} [Thu Aug 18 16:53:49 2022] POST /identity/v3/users => generated 293 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 16:53:49.477965 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08eeecc-e337-4fe5-ba7c-705bc3d01f8d None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:49.478263 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08eeecc-e337-4fe5-ba7c-705bc3d01f8d None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:49.478652 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08eeecc-e337-4fe5-ba7c-705bc3d01f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:49.498392 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-c08eeecc-e337-4fe5-ba7c-705bc3d01f8d None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:53:49.499832 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-c08eeecc-e337-4fe5-ba7c-705bc3d01f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:53:49.548951 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107dad32-bb82-44aa-8fcf-5694830eeb0b None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:49.554011 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:49.560296 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:49.586937 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 140/282] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:53:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:53:49.586937 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08eeecc-e337-4fe5-ba7c-705bc3d01f8d None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:49.601793 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f446065c-c457-4c22-8893-07c0dc056068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:49.602435 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f446065c-c457-4c22-8893-07c0dc056068 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:49.608214 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f446065c-c457-4c22-8893-07c0dc056068 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:49.608214 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f446065c-c457-4c22-8893-07c0dc056068 None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:49.608214 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f446065c-c457-4c22-8893-07c0dc056068 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:53:49.608214 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f446065c-c457-4c22-8893-07c0dc056068 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:49.608214 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f446065c-c457-4c22-8893-07c0dc056068 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:49.618585 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 143/283] 10.222.0.82 () {56 vars in 1101 bytes} [Thu Aug 18 16:53:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:49.624560 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f446065c-c457-4c22-8893-07c0dc056068 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:49.712376 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08eeecc-e337-4fe5-ba7c-705bc3d01f8d None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:49.759396 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119b2026-ba4e-432b-8314-f63f2a9bfc41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:49.760099 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119b2026-ba4e-432b-8314-f63f2a9bfc41 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:49.760362 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119b2026-ba4e-432b-8314-f63f2a9bfc41 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:49.760637 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119b2026-ba4e-432b-8314-f63f2a9bfc41 None admin] PATH_INFO: `/v3/users/2cf7e59fb28b425c861e951795fa6c61` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:49.764448 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f446065c-c457-4c22-8893-07c0dc056068 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:49.792090 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119b2026-ba4e-432b-8314-f63f2a9bfc41 None admin] RBAC: Authorizing `identity:get_user(user_id=2cf7e59fb28b425c861e951795fa6c61)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:49.793767 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119b2026-ba4e-432b-8314-f63f2a9bfc41 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:49.798966 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 141/284] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 16:53:49 2022] GET /identity/v3/users/2cf7e59fb28b425c861e951795fa6c61 => generated 293 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:49.802619 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119b2026-ba4e-432b-8314-f63f2a9bfc41 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:49.811817 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3668a5-7dfa-4161-b3b2-4d816368cb6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:49.812757 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3668a5-7dfa-4161-b3b2-4d816368cb6a None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:49.814016 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3668a5-7dfa-4161-b3b2-4d816368cb6a None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:49.814532 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3668a5-7dfa-4161-b3b2-4d816368cb6a None admin] PATH_INFO: `/v3/users/a19066a0433a4376bafbbe454b4f7079` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:49.815912 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3668a5-7dfa-4161-b3b2-4d816368cb6a None admin] RBAC: Authorizing `identity:get_user(user_id=a19066a0433a4376bafbbe454b4f7079)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:49.817568 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3668a5-7dfa-4161-b3b2-4d816368cb6a None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:49.819634 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 144/285] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 16:53:49 2022] GET /identity/v3/users/a19066a0433a4376bafbbe454b4f7079 => generated 297 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:49.825292 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3668a5-7dfa-4161-b3b2-4d816368cb6a None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:49.855961 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:49.878928 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:49.927855 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119b2026-ba4e-432b-8314-f63f2a9bfc41 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:49.948386 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3668a5-7dfa-4161-b3b2-4d816368cb6a None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:49.967681 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fed71d-b7da-496a-b90b-71efe92c6584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:49.968467 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fed71d-b7da-496a-b90b-71efe92c6584 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:49.968750 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fed71d-b7da-496a-b90b-71efe92c6584 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:49.969027 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fed71d-b7da-496a-b90b-71efe92c6584 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:49.969984 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fed71d-b7da-496a-b90b-71efe92c6584 None admin] RBAC: Adding query filter params (role.id=36d6d323d9e64e659591dbaea3081bcc, user.id=2cf7e59fb28b425c861e951795fa6c61, scope.system=all) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:53:49.970291 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fed71d-b7da-496a-b90b-71efe92c6584 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:49.971794 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fed71d-b7da-496a-b90b-71efe92c6584 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:49.977812 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 142/286] 10.222.0.82 () {56 vars in 1308 bytes} [Thu Aug 18 16:53:49 2022] GET /identity/v3/role_assignments?user.id=2cf7e59fb28b425c861e951795fa6c61&scope.system=all&role.id=36d6d323d9e64e659591dbaea3081bcc => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:53:49.978637 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fed71d-b7da-496a-b90b-71efe92c6584 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:49.994966 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a336184c-4671-433d-bf9d-1fc5e56c30ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:49.996925 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a336184c-4671-433d-bf9d-1fc5e56c30ac None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:49.997431 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a336184c-4671-433d-bf9d-1fc5e56c30ac None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:49.997883 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a336184c-4671-433d-bf9d-1fc5e56c30ac None admin] PATH_INFO: `/v3/projects/d5b629fedcc945bf8c55bda4ceedc4ae` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:49.999505 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a336184c-4671-433d-bf9d-1fc5e56c30ac None admin] RBAC: Authorizing `identity:get_project(project_id=d5b629fedcc945bf8c55bda4ceedc4ae)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:50.001839 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a336184c-4671-433d-bf9d-1fc5e56c30ac None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:50.005130 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 145/287] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 16:53:49 2022] GET /identity/v3/projects/d5b629fedcc945bf8c55bda4ceedc4ae => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:53:50.011305 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a336184c-4671-433d-bf9d-1fc5e56c30ac None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:50.138633 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a336184c-4671-433d-bf9d-1fc5e56c30ac None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:50.151768 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:50.184725 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b42fb70-5511-43f1-9e52-64aed2cb736f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:50.185698 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b42fb70-5511-43f1-9e52-64aed2cb736f None admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:50.186261 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b42fb70-5511-43f1-9e52-64aed2cb736f None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:50.186724 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b42fb70-5511-43f1-9e52-64aed2cb736f None admin] PATH_INFO: `/v3/projects/d5b629fedcc945bf8c55bda4ceedc4ae/users/a19066a0433a4376bafbbe454b4f7079/roles/d49353712e134598b13b44d21b1e791e` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:50.189173 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b42fb70-5511-43f1-9e52-64aed2cb736f None admin] RBAC: Authorizing `identity:create_grant(project_id=d5b629fedcc945bf8c55bda4ceedc4ae, user_id=a19066a0433a4376bafbbe454b4f7079, role_id=d49353712e134598b13b44d21b1e791e)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:50.191637 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b42fb70-5511-43f1-9e52-64aed2cb736f None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:50.223368 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fed71d-b7da-496a-b90b-71efe92c6584 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:50.224963 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:50.259352 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb606f2f-2bd8-4c37-997c-7ee85874fd36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:50.260071 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb606f2f-2bd8-4c37-997c-7ee85874fd36 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:50.260354 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb606f2f-2bd8-4c37-997c-7ee85874fd36 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:50.260629 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb606f2f-2bd8-4c37-997c-7ee85874fd36 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:50.272080 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb606f2f-2bd8-4c37-997c-7ee85874fd36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:53:50.277111 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-bb606f2f-2bd8-4c37-997c-7ee85874fd36 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 16:53:50.277111 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:53:50.277111 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:53:50.277111 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:53:50.277111 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:53:50.277111 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:53:50.277111 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:53:50.277111 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:53:50.277111 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:53:50.277111 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:53:50.277111 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:53:50.277111 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:53:50.277111 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:53:50.277111 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:53:50.277111 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:53:50.277111 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:53:50.280325 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:53:50.280325 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:53:50.280325 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:53:50.280325 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:53:50.280325 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:53:50.280325 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:53:50.280325 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:53:50.280325 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:53:50.280325 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:53:50.280325 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:53:50.280325 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:53:50.280325 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:53:50.280325 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:53:50.280325 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:53:50.280325 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:53:50.280325 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:53:50.281141 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:53:50.281141 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:53:50.281141 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:53:50.281141 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:53:50.281141 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:53:50.281141 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:53:50.281141 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:53:50.281141 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:53:50.281141 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:53:50.281141 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:53:50.281141 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 16:53:50.281141 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:53:50.281141 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 143/288] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:53:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:53:50.285124 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb606f2f-2bd8-4c37-997c-7ee85874fd36 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:50.311153 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 146/289] 10.222.0.82 () {58 vars in 1439 bytes} [Thu Aug 18 16:53:50 2022] PUT /identity/v3/projects/d5b629fedcc945bf8c55bda4ceedc4ae/users/a19066a0433a4376bafbbe454b4f7079/roles/d49353712e134598b13b44d21b1e791e => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 16:53:50.435254 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb606f2f-2bd8-4c37-997c-7ee85874fd36 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:50.488899 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba6d8b9-6d4f-474d-b38a-0afc3e623eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:50.489599 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba6d8b9-6d4f-474d-b38a-0afc3e623eb2 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:50.489885 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba6d8b9-6d4f-474d-b38a-0afc3e623eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:50.490156 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba6d8b9-6d4f-474d-b38a-0afc3e623eb2 None admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:50.490837 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba6d8b9-6d4f-474d-b38a-0afc3e623eb2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:53:50.491167 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba6d8b9-6d4f-474d-b38a-0afc3e623eb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:50.492721 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba6d8b9-6d4f-474d-b38a-0afc3e623eb2 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:50.508273 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 144/290] 10.222.0.82 () {56 vars in 1101 bytes} [Thu Aug 18 16:53:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:53:50.514106 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b42fb70-5511-43f1-9e52-64aed2cb736f None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:50.629555 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b42fb70-5511-43f1-9e52-64aed2cb736f None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:50.664538 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad276d0-cf49-4634-bbee-31ebf184c945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:50.665551 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad276d0-cf49-4634-bbee-31ebf184c945 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:50.666133 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad276d0-cf49-4634-bbee-31ebf184c945 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:50.666618 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad276d0-cf49-4634-bbee-31ebf184c945 None admin] PATH_INFO: `/v3/users/8fc79a3f85464a8685c64e77f557f05f` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:50.667979 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad276d0-cf49-4634-bbee-31ebf184c945 None admin] RBAC: Authorizing `identity:get_user(user_id=8fc79a3f85464a8685c64e77f557f05f)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:50.669623 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad276d0-cf49-4634-bbee-31ebf184c945 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:50.671742 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 147/291] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 16:53:50 2022] GET /identity/v3/users/8fc79a3f85464a8685c64e77f557f05f => generated 297 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:50.677237 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba6d8b9-6d4f-474d-b38a-0afc3e623eb2 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:50.711990 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:50.807157 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba6d8b9-6d4f-474d-b38a-0afc3e623eb2 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:50.861562 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a965d4-85fc-4230-b6dc-8e792dc50894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:50.862187 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a965d4-85fc-4230-b6dc-8e792dc50894 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:50.862470 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a965d4-85fc-4230-b6dc-8e792dc50894 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:50.862747 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a965d4-85fc-4230-b6dc-8e792dc50894 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:50.961029 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a965d4-85fc-4230-b6dc-8e792dc50894 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:50.962551 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a965d4-85fc-4230-b6dc-8e792dc50894 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:50.995299 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:51.073856 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 145/292] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:53:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 18 16:53:51.079516 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad276d0-cf49-4634-bbee-31ebf184c945 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:51.142564 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:51.232996 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad276d0-cf49-4634-bbee-31ebf184c945 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:51.273201 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b956a6-4dd3-4a01-868c-c6009f951ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:51.273837 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b956a6-4dd3-4a01-868c-c6009f951ed6 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:51.274120 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b956a6-4dd3-4a01-868c-c6009f951ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:51.274397 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b956a6-4dd3-4a01-868c-c6009f951ed6 None admin] PATH_INFO: `/v3/projects/d5b629fedcc945bf8c55bda4ceedc4ae` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:51.275538 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b956a6-4dd3-4a01-868c-c6009f951ed6 None admin] RBAC: Authorizing `identity:get_project(project_id=d5b629fedcc945bf8c55bda4ceedc4ae)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:51.276979 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b956a6-4dd3-4a01-868c-c6009f951ed6 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:51.278623 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 148/293] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 16:53:51 2022] GET /identity/v3/projects/d5b629fedcc945bf8c55bda4ceedc4ae => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:51.285377 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a965d4-85fc-4230-b6dc-8e792dc50894 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:51.399623 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a965d4-85fc-4230-b6dc-8e792dc50894 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:51.457834 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:51.469703 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919e933a-8ce9-465d-9716-b0d3800a99be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:51.470493 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e933a-8ce9-465d-9716-b0d3800a99be None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:51.470878 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e933a-8ce9-465d-9716-b0d3800a99be None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:51.471187 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e933a-8ce9-465d-9716-b0d3800a99be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:51.471947 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919e933a-8ce9-465d-9716-b0d3800a99be None admin] RBAC: Adding query filter params (role.id=36d6d323d9e64e659591dbaea3081bcc, scope.project.id=d5b629fedcc945bf8c55bda4ceedc4ae, user.id=8fc79a3f85464a8685c64e77f557f05f) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:53:51.472263 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919e933a-8ce9-465d-9716-b0d3800a99be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:51.473722 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919e933a-8ce9-465d-9716-b0d3800a99be None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:51.479714 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 146/294] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 16:53:51 2022] GET /identity/v3/role_assignments?user.id=8fc79a3f85464a8685c64e77f557f05f&scope.project.id=d5b629fedcc945bf8c55bda4ceedc4ae&role.id=36d6d323d9e64e659591dbaea3081bcc => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:52.041993 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236281ab-387f-4b69-973c-5c0dd06d312a None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:52.041993 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236281ab-387f-4b69-973c-5c0dd06d312a None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:52.041993 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236281ab-387f-4b69-973c-5c0dd06d312a None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:52.041993 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 149/295] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:53:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:53:52.047634 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a30c5-ddf2-47bb-9215-a377ad79ccaa None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:52.048276 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a30c5-ddf2-47bb-9215-a377ad79ccaa None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:52.048755 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a30c5-ddf2-47bb-9215-a377ad79ccaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:52.078108 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-508a30c5-ddf2-47bb-9215-a377ad79ccaa None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:53:52.080235 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-508a30c5-ddf2-47bb-9215-a377ad79ccaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:53:52.145892 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9335d1a-2bf5-467d-bb51-1180da4ec400 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:52.146509 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9335d1a-2bf5-467d-bb51-1180da4ec400 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:52.147025 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9335d1a-2bf5-467d-bb51-1180da4ec400 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:52.148567 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 150/296] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:53:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 16:53:52.156405 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecc7f21-1d9f-45f5-b650-0ade0a6b8b85 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:52.157524 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecc7f21-1d9f-45f5-b650-0ade0a6b8b85 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:52.158032 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecc7f21-1d9f-45f5-b650-0ade0a6b8b85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:52.189649 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-3ecc7f21-1d9f-45f5-b650-0ade0a6b8b85 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:53:52.192722 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-3ecc7f21-1d9f-45f5-b650-0ade0a6b8b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:53:52.208996 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 147/297] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:53:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:53:52.283522 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 151/298] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:53:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:53:52.495783 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508a30c5-ddf2-47bb-9215-a377ad79ccaa None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:52.557547 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecc7f21-1d9f-45f5-b650-0ade0a6b8b85 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:52.594327 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:52.603398 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508a30c5-ddf2-47bb-9215-a377ad79ccaa None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:52.623003 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:52.641625 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8bc69b-3be0-458a-9428-55815a5d363a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:52.642532 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8bc69b-3be0-458a-9428-55815a5d363a None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:52.642817 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8bc69b-3be0-458a-9428-55815a5d363a None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:52.643145 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8bc69b-3be0-458a-9428-55815a5d363a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:52.653884 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8bc69b-3be0-458a-9428-55815a5d363a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:53:52.658813 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-da8bc69b-3be0-458a-9428-55815a5d363a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 16:53:52.658813 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:53:52.658813 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:53:52.658813 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:53:52.658813 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:53:52.658813 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:53:52.658813 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:53:52.658813 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:53:52.658813 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:53:52.658813 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:53:52.658813 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:53:52.658813 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:53:52.658813 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:53:52.658813 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:53:52.658813 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:53:52.658813 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:53:52.659856 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:53:52.659856 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:53:52.659856 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:53:52.659856 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:53:52.659856 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:53:52.659856 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:53:52.659856 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:53:52.659856 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:53:52.659856 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:53:52.659856 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:53:52.659856 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:53:52.659856 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:53:52.659856 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:53:52.659856 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:53:52.659856 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:53:52.659856 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:53:52.660866 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:53:52.660866 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:53:52.660866 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:53:52.660866 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:53:52.660866 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:53:52.660866 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:53:52.660866 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:53:52.660866 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:53:52.660866 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:53:52.660866 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:53:52.660866 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 16:53:52.660866 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:53:52.663051 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 148/299] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:53:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:53:52.664650 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d163f10b-5121-413b-bf5d-5e75ba468884 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:52.665317 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d163f10b-5121-413b-bf5d-5e75ba468884 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:52.665599 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d163f10b-5121-413b-bf5d-5e75ba468884 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:52.669679 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecc7f21-1d9f-45f5-b650-0ade0a6b8b85 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:52.669795 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 149/300] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:53:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:53:52.669795 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d163f10b-5121-413b-bf5d-5e75ba468884 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:52.706024 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef912f6-df03-48b1-992f-1ec4057e91b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:52.706855 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef912f6-df03-48b1-992f-1ec4057e91b8 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:52.707161 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef912f6-df03-48b1-992f-1ec4057e91b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:52.707433 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef912f6-df03-48b1-992f-1ec4057e91b8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:52.708527 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef912f6-df03-48b1-992f-1ec4057e91b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:52.709961 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef912f6-df03-48b1-992f-1ec4057e91b8 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:52.714948 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 152/301] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:53:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:53:52.714948 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60fef7a-0405-4124-b8d8-1d26fd0ae953 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:52.714948 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60fef7a-0405-4124-b8d8-1d26fd0ae953 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:52.714948 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60fef7a-0405-4124-b8d8-1d26fd0ae953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:52.732894 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-d60fef7a-0405-4124-b8d8-1d26fd0ae953 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:53:52.734071 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-d60fef7a-0405-4124-b8d8-1d26fd0ae953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:53:52.779274 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d163f10b-5121-413b-bf5d-5e75ba468884 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:52.805664 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 153/302] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:53:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:53:52.806495 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60fef7a-0405-4124-b8d8-1d26fd0ae953 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:52.814615 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49af00ae-1680-4183-8c4c-dab91ba72fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:52.815255 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af00ae-1680-4183-8c4c-dab91ba72fc3 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:52.815534 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af00ae-1680-4183-8c4c-dab91ba72fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:52.815811 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af00ae-1680-4183-8c4c-dab91ba72fc3 None admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:52.816462 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49af00ae-1680-4183-8c4c-dab91ba72fc3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:53:52.817048 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49af00ae-1680-4183-8c4c-dab91ba72fc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:52.818133 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49af00ae-1680-4183-8c4c-dab91ba72fc3 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:52.832181 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 150/303] 10.222.0.82 () {56 vars in 1101 bytes} [Thu Aug 18 16:53:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:52.832181 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4158879-aa88-4361-b0fb-d75dbc3d0175 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:52.832181 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4158879-aa88-4361-b0fb-d75dbc3d0175 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:52.832181 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4158879-aa88-4361-b0fb-d75dbc3d0175 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:52.832181 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 151/304] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:53:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:53:52.835228 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4158879-aa88-4361-b0fb-d75dbc3d0175 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:52.843771 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:52.879393 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:52.917324 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60fef7a-0405-4124-b8d8-1d26fd0ae953 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:52.945088 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4158879-aa88-4361-b0fb-d75dbc3d0175 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:52.955655 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba77aa8-0453-442f-860d-77a998342749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:52.956397 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba77aa8-0453-442f-860d-77a998342749 None admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:52.956674 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba77aa8-0453-442f-860d-77a998342749 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:52.956939 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba77aa8-0453-442f-860d-77a998342749 None admin] PATH_INFO: `/v3/groups` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:52.957555 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba77aa8-0453-442f-860d-77a998342749 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:52.959003 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba77aa8-0453-442f-860d-77a998342749 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:52.979580 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb01809f-81bf-47a5-9a82-a3290c8d89a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:52.980204 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb01809f-81bf-47a5-9a82-a3290c8d89a1 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:52.980484 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb01809f-81bf-47a5-9a82-a3290c8d89a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:52.981397 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb01809f-81bf-47a5-9a82-a3290c8d89a1 None admin] PATH_INFO: `/v3/users/e0de583f5ca24744930b8dea0e3a1263` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:52.993690 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb01809f-81bf-47a5-9a82-a3290c8d89a1 None admin] RBAC: Authorizing `identity:get_user(user_id=e0de583f5ca24744930b8dea0e3a1263)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:52.995163 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb01809f-81bf-47a5-9a82-a3290c8d89a1 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:52.996736 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 152/305] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 16:53:52 2022] GET /identity/v3/users/e0de583f5ca24744930b8dea0e3a1263 => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:52.997855 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c465d24-b6e3-4553-a971-db00903a361f None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:52.998136 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c465d24-b6e3-4553-a971-db00903a361f None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:52.998407 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c465d24-b6e3-4553-a971-db00903a361f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:53.015472 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-4c465d24-b6e3-4553-a971-db00903a361f None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:53:53.017296 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-4c465d24-b6e3-4553-a971-db00903a361f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:53:53.033280 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 154/306] 10.222.0.82 () {60 vars in 1134 bytes} [Thu Aug 18 16:53:52 2022] POST /identity/v3/groups => generated 229 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 16:53:53.034389 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba77aa8-0453-442f-860d-77a998342749 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:53.102725 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 153/307] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:53:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:53:53.103658 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c465d24-b6e3-4553-a971-db00903a361f None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:53.171120 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:53.179344 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:53.183815 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba77aa8-0453-442f-860d-77a998342749 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:53.216184 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c465d24-b6e3-4553-a971-db00903a361f None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:53.232653 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27005040-f75c-4710-84a4-c5b15fb35649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:53.233274 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27005040-f75c-4710-84a4-c5b15fb35649 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:53.233559 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27005040-f75c-4710-84a4-c5b15fb35649 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:53.233836 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27005040-f75c-4710-84a4-c5b15fb35649 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:53.235003 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27005040-f75c-4710-84a4-c5b15fb35649 None admin] RBAC: Adding query filter params (role.id=d49353712e134598b13b44d21b1e791e, user.id=e0de583f5ca24744930b8dea0e3a1263, scope.system=all) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:53:53.235560 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27005040-f75c-4710-84a4-c5b15fb35649 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:53.236786 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27005040-f75c-4710-84a4-c5b15fb35649 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:53.244483 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 155/308] 10.222.0.82 () {56 vars in 1308 bytes} [Thu Aug 18 16:53:53 2022] GET /identity/v3/role_assignments?user.id=e0de583f5ca24744930b8dea0e3a1263&scope.system=all&role.id=d49353712e134598b13b44d21b1e791e => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:53.257404 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a56a27-b7dc-42dc-af61-b761702145d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:53.258075 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a56a27-b7dc-42dc-af61-b761702145d4 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:53.258357 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a56a27-b7dc-42dc-af61-b761702145d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:53.258637 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a56a27-b7dc-42dc-af61-b761702145d4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:53.272089 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a56a27-b7dc-42dc-af61-b761702145d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:53:53.277065 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-66a56a27-b7dc-42dc-af61-b761702145d4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 16:53:53.277065 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:53:53.277065 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:53:53.277065 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:53:53.277065 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:53:53.277065 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:53:53.277065 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:53:53.277065 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:53:53.277065 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:53:53.277065 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:53:53.277065 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:53:53.277065 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:53:53.277065 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:53:53.277065 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:53:53.277065 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:53:53.277065 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:53:53.277966 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:53:53.277966 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:53:53.277966 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:53:53.277966 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:53:53.277966 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:53:53.277966 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:53:53.277966 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:53:53.277966 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:53:53.277966 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:53:53.277966 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:53:53.277966 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:53:53.277966 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:53:53.277966 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:53:53.277966 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:53:53.277966 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:53:53.277966 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:53:53.278722 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:53:53.278722 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:53:53.278722 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:53:53.278722 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:53:53.278722 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:53:53.278722 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:53:53.278722 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:53:53.278722 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:53:53.278722 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:53:53.278722 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:53:53.278722 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 16:53:53.278722 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:53:53.278722 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 154/309] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:53:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:53:53.286386 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27005040-f75c-4710-84a4-c5b15fb35649 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:53.387397 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a56a27-b7dc-42dc-af61-b761702145d4 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:53.398603 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27005040-f75c-4710-84a4-c5b15fb35649 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:53.442078 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300616a7-0722-40e7-bab6-88e1e6ca8b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:53.443159 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300616a7-0722-40e7-bab6-88e1e6ca8b22 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:53.443638 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300616a7-0722-40e7-bab6-88e1e6ca8b22 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:53.443910 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300616a7-0722-40e7-bab6-88e1e6ca8b22 None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:53.444574 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300616a7-0722-40e7-bab6-88e1e6ca8b22 None admin] RBAC: Adding query filter params (name=member) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:53:53.444870 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300616a7-0722-40e7-bab6-88e1e6ca8b22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:53.446412 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300616a7-0722-40e7-bab6-88e1e6ca8b22 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:53.460678 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 156/310] 10.222.0.82 () {56 vars in 1101 bytes} [Thu Aug 18 16:53:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:53.462069 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300616a7-0722-40e7-bab6-88e1e6ca8b22 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:53.483865 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:53.500720 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:53.513120 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a56a27-b7dc-42dc-af61-b761702145d4 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:53.563192 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c871e1-444d-47df-ab11-89e758d0459b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:53.564206 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c871e1-444d-47df-ab11-89e758d0459b None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:53.564703 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c871e1-444d-47df-ab11-89e758d0459b None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:53.564985 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c871e1-444d-47df-ab11-89e758d0459b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:53.579954 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c871e1-444d-47df-ab11-89e758d0459b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:53:53.580823 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300616a7-0722-40e7-bab6-88e1e6ca8b22 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:53.587182 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-32c871e1-444d-47df-ab11-89e758d0459b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 16:53:53.587182 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:53:53.587182 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:53:53.587182 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:53:53.587182 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:53:53.587182 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:53:53.587182 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:53:53.587182 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:53:53.587182 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:53:53.587182 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:53:53.587182 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:53:53.587182 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:53:53.587182 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:53:53.587182 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:53:53.587182 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:53:53.587182 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:53:53.588039 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:53:53.588039 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:53:53.588039 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:53:53.588039 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:53:53.588039 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:53:53.588039 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:53:53.588039 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:53:53.588039 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:53:53.588039 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:53:53.588039 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:53:53.588039 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:53:53.588039 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:53:53.588039 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:53:53.588039 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:53:53.588039 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:53:53.588039 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:53:53.588771 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:53:53.588771 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:53:53.588771 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:53:53.588771 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:53:53.588771 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:53:53.588771 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:53:53.588771 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:53:53.588771 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:53:53.588771 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:53:53.588771 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:53:53.588771 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 16:53:53.588771 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:53:53.591051 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 155/311] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:53:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:53:53.596654 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c871e1-444d-47df-ab11-89e758d0459b None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:53.624761 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b603e18d-7661-4a8c-91aa-fc2a00961884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:53.626179 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b603e18d-7661-4a8c-91aa-fc2a00961884 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:53.626560 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b603e18d-7661-4a8c-91aa-fc2a00961884 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:53.627011 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b603e18d-7661-4a8c-91aa-fc2a00961884 None admin] PATH_INFO: `/v3/users/2cf7e59fb28b425c861e951795fa6c61` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:53.628304 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b603e18d-7661-4a8c-91aa-fc2a00961884 None admin] RBAC: Authorizing `identity:get_user(user_id=2cf7e59fb28b425c861e951795fa6c61)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:53.629927 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b603e18d-7661-4a8c-91aa-fc2a00961884 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:53.632060 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 157/312] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 16:53:53 2022] GET /identity/v3/users/2cf7e59fb28b425c861e951795fa6c61 => generated 293 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:53.637293 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b603e18d-7661-4a8c-91aa-fc2a00961884 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:53.713716 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c871e1-444d-47df-ab11-89e758d0459b None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:53.748606 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7498f4d6-308b-4bcd-aa0c-8b5f44d3fb2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:53.749223 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498f4d6-308b-4bcd-aa0c-8b5f44d3fb2d None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:53.749505 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498f4d6-308b-4bcd-aa0c-8b5f44d3fb2d None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:53.749779 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498f4d6-308b-4bcd-aa0c-8b5f44d3fb2d None admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:53.751199 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7498f4d6-308b-4bcd-aa0c-8b5f44d3fb2d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:53:53.751568 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7498f4d6-308b-4bcd-aa0c-8b5f44d3fb2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:53.752612 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:53.753903 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7498f4d6-308b-4bcd-aa0c-8b5f44d3fb2d None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:53.763920 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 156/313] 10.222.0.82 () {56 vars in 1101 bytes} [Thu Aug 18 16:53:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:53.770454 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7498f4d6-308b-4bcd-aa0c-8b5f44d3fb2d None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:53.784644 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b603e18d-7661-4a8c-91aa-fc2a00961884 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:53.790309 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:53.823792 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d83214-8ca8-41da-83d8-5f2bf7124bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:53.824361 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d83214-8ca8-41da-83d8-5f2bf7124bb5 None admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:53.824640 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d83214-8ca8-41da-83d8-5f2bf7124bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:53.824915 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d83214-8ca8-41da-83d8-5f2bf7124bb5 None admin] PATH_INFO: `/v3/system/users/2cf7e59fb28b425c861e951795fa6c61/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:53.826394 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d83214-8ca8-41da-83d8-5f2bf7124bb5 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2cf7e59fb28b425c861e951795fa6c61, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:53.829263 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d83214-8ca8-41da-83d8-5f2bf7124bb5 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:53.891642 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7498f4d6-308b-4bcd-aa0c-8b5f44d3fb2d None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:53.905207 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 158/314] 10.222.0.82 () {58 vars in 1334 bytes} [Thu Aug 18 16:53:53 2022] PUT /identity/v3/system/users/2cf7e59fb28b425c861e951795fa6c61/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 16:53:53.933871 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad19462-b1a3-4225-8a21-83cdb5ea7d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:53.935069 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad19462-b1a3-4225-8a21-83cdb5ea7d76 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:53.935646 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad19462-b1a3-4225-8a21-83cdb5ea7d76 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:53.936196 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad19462-b1a3-4225-8a21-83cdb5ea7d76 None admin] PATH_INFO: `/v3/users/a19066a0433a4376bafbbe454b4f7079` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:53.937905 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad19462-b1a3-4225-8a21-83cdb5ea7d76 None admin] RBAC: Authorizing `identity:get_user(user_id=a19066a0433a4376bafbbe454b4f7079)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:53.939761 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad19462-b1a3-4225-8a21-83cdb5ea7d76 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:53.942457 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 157/315] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 16:53:53 2022] GET /identity/v3/users/a19066a0433a4376bafbbe454b4f7079 => generated 297 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:53.947933 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d83214-8ca8-41da-83d8-5f2bf7124bb5 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:54.062541 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d83214-8ca8-41da-83d8-5f2bf7124bb5 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:54.100285 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb2f61c-0e46-4a18-9ac4-ffbf3058ffb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:54.100820 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2f61c-0e46-4a18-9ac4-ffbf3058ffb0 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:54.101108 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2f61c-0e46-4a18-9ac4-ffbf3058ffb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:54.101381 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2f61c-0e46-4a18-9ac4-ffbf3058ffb0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:54.120108 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:54.172151 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb2f61c-0e46-4a18-9ac4-ffbf3058ffb0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:54.173629 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb2f61c-0e46-4a18-9ac4-ffbf3058ffb0 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:54.277145 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 159/316] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:53:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 18 16:53:54.282364 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad19462-b1a3-4225-8a21-83cdb5ea7d76 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:54.365787 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:54.390336 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad19462-b1a3-4225-8a21-83cdb5ea7d76 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:54.428010 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b4a51c-237c-4946-b44c-e97ced9c1eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:54.429004 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b4a51c-237c-4946-b44c-e97ced9c1eb6 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:54.429504 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b4a51c-237c-4946-b44c-e97ced9c1eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:54.429971 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b4a51c-237c-4946-b44c-e97ced9c1eb6 None admin] PATH_INFO: `/v3/projects/d5b629fedcc945bf8c55bda4ceedc4ae` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:54.431430 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b4a51c-237c-4946-b44c-e97ced9c1eb6 None admin] RBAC: Authorizing `identity:get_project(project_id=d5b629fedcc945bf8c55bda4ceedc4ae)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:54.433121 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b4a51c-237c-4946-b44c-e97ced9c1eb6 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:54.434962 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 158/317] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 16:53:54 2022] GET /identity/v3/projects/d5b629fedcc945bf8c55bda4ceedc4ae => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:53:54.441596 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb2f61c-0e46-4a18-9ac4-ffbf3058ffb0 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:54.494011 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:54.549050 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb2f61c-0e46-4a18-9ac4-ffbf3058ffb0 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:54.583399 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdd0315-a3d1-4ff4-ac30-92732bcfb048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:54.584004 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd0315-a3d1-4ff4-ac30-92732bcfb048 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:54.584283 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd0315-a3d1-4ff4-ac30-92732bcfb048 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:54.584556 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd0315-a3d1-4ff4-ac30-92732bcfb048 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:54.585272 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdd0315-a3d1-4ff4-ac30-92732bcfb048 None admin] RBAC: Adding query filter params (role.id=d49353712e134598b13b44d21b1e791e, scope.project.id=d5b629fedcc945bf8c55bda4ceedc4ae, user.id=a19066a0433a4376bafbbe454b4f7079) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:53:54.585579 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdd0315-a3d1-4ff4-ac30-92732bcfb048 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:54.587017 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdd0315-a3d1-4ff4-ac30-92732bcfb048 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:54.593302 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 160/318] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 16:53:54 2022] GET /identity/v3/role_assignments?user.id=a19066a0433a4376bafbbe454b4f7079&scope.project.id=d5b629fedcc945bf8c55bda4ceedc4ae&role.id=d49353712e134598b13b44d21b1e791e => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:55.087260 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d320664-5381-47f0-bcc2-e7ec79597283 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:55.087768 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d320664-5381-47f0-bcc2-e7ec79597283 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:55.087871 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d320664-5381-47f0-bcc2-e7ec79597283 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:55.089169 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 159/319] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:53:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:53:55.095584 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f40f569-ff9c-4708-8531-655acd9551e7 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:55.095872 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f40f569-ff9c-4708-8531-655acd9551e7 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:55.096148 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f40f569-ff9c-4708-8531-655acd9551e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:55.116059 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-0f40f569-ff9c-4708-8531-655acd9551e7 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:53:55.117268 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-0f40f569-ff9c-4708-8531-655acd9551e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:53:55.187753 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 161/320] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:53:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:53:55.239250 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a0746e-6fec-4277-a099-b5cafca6cf57 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:55.239544 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a0746e-6fec-4277-a099-b5cafca6cf57 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:55.239824 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a0746e-6fec-4277-a099-b5cafca6cf57 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:55.241304 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 160/321] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:53:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:53:55.246227 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f37b51-b4ec-4e3b-9655-2bb7eaae0d3f None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:55.246523 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f37b51-b4ec-4e3b-9655-2bb7eaae0d3f None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:55.246805 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f37b51-b4ec-4e3b-9655-2bb7eaae0d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:55.264414 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-92f37b51-b4ec-4e3b-9655-2bb7eaae0d3f None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:53:55.265587 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-92f37b51-b4ec-4e3b-9655-2bb7eaae0d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:53:55.305395 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:55.331517 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 162/322] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:53:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:53:55.461208 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a0746e-6fec-4277-a099-b5cafca6cf57 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:55.565472 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a0746e-6fec-4277-a099-b5cafca6cf57 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:55.598498 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f37b51-b4ec-4e3b-9655-2bb7eaae0d3f None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:55.599934 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ed4d7a-3667-49b2-8539-6aaafcd76366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:55.600572 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed4d7a-3667-49b2-8539-6aaafcd76366 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:55.600910 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed4d7a-3667-49b2-8539-6aaafcd76366 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:55.601185 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed4d7a-3667-49b2-8539-6aaafcd76366 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:55.602272 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ed4d7a-3667-49b2-8539-6aaafcd76366 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:55.603746 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ed4d7a-3667-49b2-8539-6aaafcd76366 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:55.605433 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 161/323] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:53:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:55.611503 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ed4d7a-3667-49b2-8539-6aaafcd76366 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:55.628036 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:55.708890 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f37b51-b4ec-4e3b-9655-2bb7eaae0d3f None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:55.715794 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ed4d7a-3667-49b2-8539-6aaafcd76366 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:55.745862 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7090c070-eebc-448b-be2a-6c7c7e986666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:55.746508 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7090c070-eebc-448b-be2a-6c7c7e986666 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:55.746792 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7090c070-eebc-448b-be2a-6c7c7e986666 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:55.747096 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7090c070-eebc-448b-be2a-6c7c7e986666 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:55.752394 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d2350f-61d6-4c44-9d75-4e257a39582e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:55.752999 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d2350f-61d6-4c44-9d75-4e257a39582e None admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:55.753276 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d2350f-61d6-4c44-9d75-4e257a39582e None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:55.753546 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d2350f-61d6-4c44-9d75-4e257a39582e None admin] PATH_INFO: `/v3/groups` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:55.754162 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d2350f-61d6-4c44-9d75-4e257a39582e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:55.755639 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d2350f-61d6-4c44-9d75-4e257a39582e None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:55.757837 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7090c070-eebc-448b-be2a-6c7c7e986666 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:53:55.764950 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-7090c070-eebc-448b-be2a-6c7c7e986666 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 16:53:55.764950 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:53:55.764950 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:53:55.764950 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:53:55.764950 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:53:55.764950 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:53:55.764950 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:53:55.764950 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:53:55.764950 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:53:55.764950 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:53:55.764950 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:53:55.764950 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:53:55.764950 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:53:55.764950 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:53:55.764950 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:53:55.764950 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:53:55.768324 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:53:55.768324 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:53:55.768324 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:53:55.768324 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:53:55.768324 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:53:55.768324 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:53:55.768324 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:53:55.768324 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:53:55.768324 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:53:55.768324 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:53:55.768324 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:53:55.768324 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:53:55.768324 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:53:55.768324 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:53:55.768324 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:53:55.768324 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:53:55.769866 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:53:55.769866 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:53:55.769866 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:53:55.769866 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:53:55.769866 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:53:55.769866 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:53:55.769866 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:53:55.769866 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:53:55.769866 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:53:55.769866 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:53:55.769866 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 16:53:55.769866 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:53:55.769866 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 163/324] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:53:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:53:55.776457 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7090c070-eebc-448b-be2a-6c7c7e986666 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:55.821467 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:55.852301 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 162/325] 10.222.0.82 () {60 vars in 1134 bytes} [Thu Aug 18 16:53:55 2022] POST /identity/v3/groups => generated 226 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 16:53:55.889842 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7090c070-eebc-448b-be2a-6c7c7e986666 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:55.928148 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2fbd4b-c3f6-43a9-a177-6f7ff70db8ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:55.928783 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2fbd4b-c3f6-43a9-a177-6f7ff70db8ae None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:55.929047 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2fbd4b-c3f6-43a9-a177-6f7ff70db8ae None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:55.929382 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2fbd4b-c3f6-43a9-a177-6f7ff70db8ae None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:55.930084 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2fbd4b-c3f6-43a9-a177-6f7ff70db8ae None admin] RBAC: Adding query filter params (name=reader) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:53:55.930359 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2fbd4b-c3f6-43a9-a177-6f7ff70db8ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:55.931830 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2fbd4b-c3f6-43a9-a177-6f7ff70db8ae None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:55.945969 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 164/326] 10.222.0.82 () {56 vars in 1101 bytes} [Thu Aug 18 16:53:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:53:55.947303 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d2350f-61d6-4c44-9d75-4e257a39582e None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:55.957666 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143bd23e-3084-4f9d-8303-a970c7e75f27 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:55.958025 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143bd23e-3084-4f9d-8303-a970c7e75f27 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:55.958303 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143bd23e-3084-4f9d-8303-a970c7e75f27 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:55.959837 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 165/327] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:53:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 16:53:55.965248 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7974b8a-ddd5-4488-ac2c-c86fea440bab None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:55.965527 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7974b8a-ddd5-4488-ac2c-c86fea440bab None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:55.965859 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7974b8a-ddd5-4488-ac2c-c86fea440bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:55.985262 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-f7974b8a-ddd5-4488-ac2c-c86fea440bab None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:53:55.986554 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-f7974b8a-ddd5-4488-ac2c-c86fea440bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:53:56.030256 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:56.057666 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d2350f-61d6-4c44-9d75-4e257a39582e None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:56.068236 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 166/328] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:53:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 16:53:56.100461 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93edf4e-6934-4a38-9806-7837c51f4708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:56.101111 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93edf4e-6934-4a38-9806-7837c51f4708 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:56.101398 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93edf4e-6934-4a38-9806-7837c51f4708 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:56.101674 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93edf4e-6934-4a38-9806-7837c51f4708 None admin] PATH_INFO: `/v3/users/e0de583f5ca24744930b8dea0e3a1263` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:56.102847 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93edf4e-6934-4a38-9806-7837c51f4708 None admin] RBAC: Authorizing `identity:get_user(user_id=e0de583f5ca24744930b8dea0e3a1263)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:56.104390 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93edf4e-6934-4a38-9806-7837c51f4708 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:56.106057 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 163/329] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 16:53:55 2022] GET /identity/v3/users/e0de583f5ca24744930b8dea0e3a1263 => generated 293 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:56.112624 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7974b8a-ddd5-4488-ac2c-c86fea440bab None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:56.147756 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:56.230189 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7974b8a-ddd5-4488-ac2c-c86fea440bab None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:56.265432 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188cfc3e-544d-4513-85ee-71d1bb4fe69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:56.266643 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188cfc3e-544d-4513-85ee-71d1bb4fe69c None admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:56.266946 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188cfc3e-544d-4513-85ee-71d1bb4fe69c None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:56.267237 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188cfc3e-544d-4513-85ee-71d1bb4fe69c None admin] PATH_INFO: `/v3/system/users/e0de583f5ca24744930b8dea0e3a1263/roles/d49353712e134598b13b44d21b1e791e` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:56.268716 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188cfc3e-544d-4513-85ee-71d1bb4fe69c None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=e0de583f5ca24744930b8dea0e3a1263, role_id=d49353712e134598b13b44d21b1e791e)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:56.270152 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188cfc3e-544d-4513-85ee-71d1bb4fe69c None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:56.345942 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93edf4e-6934-4a38-9806-7837c51f4708 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:56.348777 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 167/330] 10.222.0.82 () {58 vars in 1334 bytes} [Thu Aug 18 16:53:56 2022] PUT /identity/v3/system/users/e0de583f5ca24744930b8dea0e3a1263/roles/d49353712e134598b13b44d21b1e791e => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 16:53:56.462634 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93edf4e-6934-4a38-9806-7837c51f4708 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:56.503386 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8814362-6524-47a6-9a12-d50cbe73ef54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:56.504272 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8814362-6524-47a6-9a12-d50cbe73ef54 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:56.504716 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8814362-6524-47a6-9a12-d50cbe73ef54 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:56.505132 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8814362-6524-47a6-9a12-d50cbe73ef54 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:56.518749 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8814362-6524-47a6-9a12-d50cbe73ef54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:53:56.524780 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-e8814362-6524-47a6-9a12-d50cbe73ef54 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 16:53:56.524780 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:53:56.524780 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:53:56.524780 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:53:56.524780 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:53:56.524780 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:53:56.524780 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:53:56.524780 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:53:56.524780 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:53:56.524780 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:53:56.524780 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:53:56.524780 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:53:56.524780 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:53:56.524780 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:53:56.524780 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:53:56.524780 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:53:56.525867 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:53:56.525867 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:53:56.525867 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:53:56.525867 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:53:56.525867 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:53:56.525867 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:53:56.525867 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:53:56.525867 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:53:56.525867 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:53:56.525867 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:53:56.525867 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:53:56.525867 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:53:56.525867 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:53:56.525867 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:53:56.525867 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:53:56.525867 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:53:56.526824 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:53:56.526824 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:53:56.526824 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:53:56.526824 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:53:56.526824 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:53:56.526824 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:53:56.526824 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:53:56.526824 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:53:56.526824 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:53:56.526824 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:53:56.526824 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 16:53:56.526824 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:53:56.529166 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 164/331] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:53:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 16:53:56.534921 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188cfc3e-544d-4513-85ee-71d1bb4fe69c None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:56.646867 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:56.656835 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188cfc3e-544d-4513-85ee-71d1bb4fe69c None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:56.694745 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ed751b-ddab-4795-a011-4ab37d34e985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:56.695609 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed751b-ddab-4795-a011-4ab37d34e985 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:56.696071 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed751b-ddab-4795-a011-4ab37d34e985 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:56.697206 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed751b-ddab-4795-a011-4ab37d34e985 None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:56.697883 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ed751b-ddab-4795-a011-4ab37d34e985 None admin] RBAC: Adding query filter params (name=member) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:53:56.698188 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ed751b-ddab-4795-a011-4ab37d34e985 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:56.699698 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ed751b-ddab-4795-a011-4ab37d34e985 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:56.709513 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 168/332] 10.222.0.82 () {56 vars in 1101 bytes} [Thu Aug 18 16:53:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:53:56.714716 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8814362-6524-47a6-9a12-d50cbe73ef54 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:56.719532 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:56.839816 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8814362-6524-47a6-9a12-d50cbe73ef54 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:56.881751 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3882cb3c-b96f-4799-99c9-180f2c92119e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:56.882394 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3882cb3c-b96f-4799-99c9-180f2c92119e None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:56.882677 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3882cb3c-b96f-4799-99c9-180f2c92119e None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:56.883001 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3882cb3c-b96f-4799-99c9-180f2c92119e None admin] PATH_INFO: `/v3/users/2cf7e59fb28b425c861e951795fa6c61` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:56.884144 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3882cb3c-b96f-4799-99c9-180f2c92119e None admin] RBAC: Authorizing `identity:get_user(user_id=2cf7e59fb28b425c861e951795fa6c61)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:56.885619 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3882cb3c-b96f-4799-99c9-180f2c92119e None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:56.887231 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 165/333] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 16:53:56 2022] GET /identity/v3/users/2cf7e59fb28b425c861e951795fa6c61 => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:56.893752 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ed751b-ddab-4795-a011-4ab37d34e985 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:57.026710 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ed751b-ddab-4795-a011-4ab37d34e985 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:57.076477 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1267ab2-0c15-44f9-a9f8-5810a5f4d0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:57.077118 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1267ab2-0c15-44f9-a9f8-5810a5f4d0fc None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:57.077401 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1267ab2-0c15-44f9-a9f8-5810a5f4d0fc None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:57.077672 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1267ab2-0c15-44f9-a9f8-5810a5f4d0fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:57.078976 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1267ab2-0c15-44f9-a9f8-5810a5f4d0fc None admin] RBAC: Adding query filter params (role.id=36d6d323d9e64e659591dbaea3081bcc, user.id=2cf7e59fb28b425c861e951795fa6c61, scope.system=all) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:53:57.079285 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1267ab2-0c15-44f9-a9f8-5810a5f4d0fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:57.080814 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1267ab2-0c15-44f9-a9f8-5810a5f4d0fc None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:57.097457 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 169/334] 10.222.0.82 () {56 vars in 1308 bytes} [Thu Aug 18 16:53:56 2022] GET /identity/v3/role_assignments?user.id=2cf7e59fb28b425c861e951795fa6c61&scope.system=all&role.id=36d6d323d9e64e659591dbaea3081bcc => generated 510 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:58.250497 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0946da-33d5-45e0-9e92-65d399927ef8 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:58.251561 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0946da-33d5-45e0-9e92-65d399927ef8 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:58.252168 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0946da-33d5-45e0-9e92-65d399927ef8 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:58.254075 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 166/335] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:53:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:53:58.269512 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1289b6ec-8480-4daf-bb03-3272a8d7eb7e None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:58.270406 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1289b6ec-8480-4daf-bb03-3272a8d7eb7e None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:58.270947 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1289b6ec-8480-4daf-bb03-3272a8d7eb7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:58.294659 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-1289b6ec-8480-4daf-bb03-3272a8d7eb7e None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:53:58.296218 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-1289b6ec-8480-4daf-bb03-3272a8d7eb7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:53:58.300309 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:58.380035 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 170/336] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:53:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:53:58.790313 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8c2b58-a01f-4754-b56b-3c4a0754ecce None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:58.790665 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8c2b58-a01f-4754-b56b-3c4a0754ecce None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:58.790868 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8c2b58-a01f-4754-b56b-3c4a0754ecce None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:58.792377 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 167/337] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:53:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:53:58.799173 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef8cc90-e9a7-43a1-8c8e-d4d7fe6bca1a None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:58.799741 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef8cc90-e9a7-43a1-8c8e-d4d7fe6bca1a None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:58.800216 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef8cc90-e9a7-43a1-8c8e-d4d7fe6bca1a None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:58.801986 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 171/338] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:53:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:53:58.802964 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe94de6-0f93-4ffa-b285-f5140369d85e None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:58.803256 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe94de6-0f93-4ffa-b285-f5140369d85e None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:58.803528 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe94de6-0f93-4ffa-b285-f5140369d85e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:58.806727 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01697599-a81e-4750-8b77-8e0a38d99a24 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:58.807271 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01697599-a81e-4750-8b77-8e0a38d99a24 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:58.807789 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01697599-a81e-4750-8b77-8e0a38d99a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:58.832363 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-cfe94de6-0f93-4ffa-b285-f5140369d85e None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:53:58.833252 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-01697599-a81e-4750-8b77-8e0a38d99a24 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:53:58.833552 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-cfe94de6-0f93-4ffa-b285-f5140369d85e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:53:58.834817 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-01697599-a81e-4750-8b77-8e0a38d99a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:53:58.936601 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:58.942446 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 168/339] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:53:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 16:53:58.944019 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe94de6-0f93-4ffa-b285-f5140369d85e None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:59.016290 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 172/340] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:53:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 16:53:59.074615 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe94de6-0f93-4ffa-b285-f5140369d85e None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:59.125113 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9bd7c4-6532-4909-adcc-c805e4a9ce1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:59.125771 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9bd7c4-6532-4909-adcc-c805e4a9ce1d None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:59.126140 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9bd7c4-6532-4909-adcc-c805e4a9ce1d None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:59.126415 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9bd7c4-6532-4909-adcc-c805e4a9ce1d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:59.136797 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9bd7c4-6532-4909-adcc-c805e4a9ce1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:53:59.142468 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-3a9bd7c4-6532-4909-adcc-c805e4a9ce1d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 16:53:59.142468 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:53:59.142468 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:53:59.142468 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:53:59.142468 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:53:59.142468 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:53:59.142468 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:53:59.142468 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:53:59.142468 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:53:59.142468 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:53:59.142468 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:53:59.142468 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:53:59.142468 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:53:59.142468 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:53:59.142468 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:53:59.142468 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:53:59.143384 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:53:59.143384 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:53:59.143384 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:53:59.143384 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:53:59.143384 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:53:59.143384 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:53:59.143384 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:53:59.143384 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:53:59.143384 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:53:59.143384 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:53:59.143384 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:53:59.143384 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:53:59.143384 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:53:59.143384 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:53:59.143384 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:53:59.143384 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:53:59.144135 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:53:59.144135 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:53:59.144135 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:53:59.144135 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:53:59.144135 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:53:59.144135 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:53:59.144135 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:53:59.144135 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:53:59.144135 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:53:59.144135 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:53:59.144135 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 16:53:59.144135 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:53:59.144135 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 169/341] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:53:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:53:59.150965 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01697599-a81e-4750-8b77-8e0a38d99a24 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:59.255248 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:59.289013 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01697599-a81e-4750-8b77-8e0a38d99a24 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:59.334790 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6895b2db-b635-44b4-9f6d-78e079de61c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:59.335359 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6895b2db-b635-44b4-9f6d-78e079de61c2 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:59.335639 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6895b2db-b635-44b4-9f6d-78e079de61c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:59.335914 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6895b2db-b635-44b4-9f6d-78e079de61c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:59.336581 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6895b2db-b635-44b4-9f6d-78e079de61c2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:53:59.336884 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6895b2db-b635-44b4-9f6d-78e079de61c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:59.338266 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6895b2db-b635-44b4-9f6d-78e079de61c2 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:59.350596 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 173/342] 10.222.0.82 () {56 vars in 1101 bytes} [Thu Aug 18 16:53:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:59.356461 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9bd7c4-6532-4909-adcc-c805e4a9ce1d None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:59.372468 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6895b2db-b635-44b4-9f6d-78e079de61c2 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:59.452990 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:59.482491 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6895b2db-b635-44b4-9f6d-78e079de61c2 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:59.515173 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:59.522273 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b84b594-0ab3-47ee-816c-b51f605cc9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:59.523185 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84b594-0ab3-47ee-816c-b51f605cc9f1 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:59.523666 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84b594-0ab3-47ee-816c-b51f605cc9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:59.524155 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84b594-0ab3-47ee-816c-b51f605cc9f1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:59.535856 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b84b594-0ab3-47ee-816c-b51f605cc9f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:53:59.540928 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-3b84b594-0ab3-47ee-816c-b51f605cc9f1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 16:53:59.540928 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:53:59.540928 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:53:59.540928 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:53:59.540928 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:53:59.540928 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:53:59.540928 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:53:59.540928 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:53:59.540928 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:53:59.540928 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:53:59.540928 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:53:59.540928 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:53:59.540928 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:53:59.540928 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:53:59.540928 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:53:59.540928 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:53:59.542006 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:53:59.542006 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:53:59.542006 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:53:59.542006 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:53:59.542006 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:53:59.542006 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:53:59.542006 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:53:59.542006 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:53:59.542006 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:53:59.542006 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:53:59.542006 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:53:59.542006 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:53:59.542006 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:53:59.542006 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:53:59.542006 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:53:59.542006 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:53:59.542796 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:53:59.542796 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:53:59.542796 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:53:59.542796 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:53:59.542796 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:53:59.542796 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:53:59.542796 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:53:59.542796 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:53:59.542796 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:53:59.542796 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:53:59.542796 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 16:53:59.542796 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:53:59.548911 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 174/343] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:53:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 16:53:59.549828 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9bd7c4-6532-4909-adcc-c805e4a9ce1d None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:59.550671 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee852d1-4c26-48f7-bdf9-01533c86c219 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:59.551239 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee852d1-4c26-48f7-bdf9-01533c86c219 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:59.551704 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee852d1-4c26-48f7-bdf9-01533c86c219 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:59.554317 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 175/344] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:53:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:53:59.557763 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee852d1-4c26-48f7-bdf9-01533c86c219 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:59.611019 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc387bd-1d89-4c76-a1bc-77842bb10392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:59.611347 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc387bd-1d89-4c76-a1bc-77842bb10392 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:59.611623 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc387bd-1d89-4c76-a1bc-77842bb10392 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:59.611897 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc387bd-1d89-4c76-a1bc-77842bb10392 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:59.680599 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee852d1-4c26-48f7-bdf9-01533c86c219 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:59.689924 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc387bd-1d89-4c76-a1bc-77842bb10392 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:59.691471 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc387bd-1d89-4c76-a1bc-77842bb10392 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:59.719271 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f9a37-4819-4a81-9dd4-e0a2862d6c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:59.720374 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f9a37-4819-4a81-9dd4-e0a2862d6c09 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:59.720872 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f9a37-4819-4a81-9dd4-e0a2862d6c09 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:59.721381 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f9a37-4819-4a81-9dd4-e0a2862d6c09 None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:59.722339 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88f9a37-4819-4a81-9dd4-e0a2862d6c09 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:53:59.722832 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88f9a37-4819-4a81-9dd4-e0a2862d6c09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:53:59.724547 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88f9a37-4819-4a81-9dd4-e0a2862d6c09 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:53:59.739281 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 176/345] 10.222.0.82 () {56 vars in 1101 bytes} [Thu Aug 18 16:53:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:53:59.740951 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0090b2a7-9983-4e81-b995-70a9c13e0932 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:59.741719 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0090b2a7-9983-4e81-b995-70a9c13e0932 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:59.742184 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0090b2a7-9983-4e81-b995-70a9c13e0932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:59.754632 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:59.775965 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-0090b2a7-9983-4e81-b995-70a9c13e0932 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:53:59.787100 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-0090b2a7-9983-4e81-b995-70a9c13e0932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:53:59.818591 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 170/346] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:53:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 16:53:59.819777 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc387bd-1d89-4c76-a1bc-77842bb10392 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:59.839184 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:53:59.858235 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 177/347] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:53:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:53:59.861192 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0090b2a7-9983-4e81-b995-70a9c13e0932 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:53:59.931260 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc387bd-1d89-4c76-a1bc-77842bb10392 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:59.966607 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0090b2a7-9983-4e81-b995-70a9c13e0932 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:53:59.970058 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95206a5e-a4e0-48d5-ba14-045b88646010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:53:59.971714 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95206a5e-a4e0-48d5-ba14-045b88646010 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:53:59.971714 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95206a5e-a4e0-48d5-ba14-045b88646010 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:53:59.971714 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95206a5e-a4e0-48d5-ba14-045b88646010 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:53:59.983177 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95206a5e-a4e0-48d5-ba14-045b88646010 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:53:59.989160 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-95206a5e-a4e0-48d5-ba14-045b88646010 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 16:53:59.989160 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:53:59.989160 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:53:59.989160 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:53:59.989160 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:53:59.989160 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:53:59.989160 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:53:59.989160 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:53:59.989160 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:53:59.989160 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:53:59.989160 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:53:59.989160 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:53:59.989160 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:53:59.989160 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:53:59.989160 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:53:59.989160 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:53:59.990123 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:53:59.990123 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:53:59.990123 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:53:59.990123 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:53:59.990123 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:53:59.990123 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:53:59.990123 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:53:59.990123 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:53:59.990123 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:53:59.990123 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:53:59.990123 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:53:59.990123 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:53:59.990123 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:53:59.990123 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:53:59.990123 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:53:59.990123 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:53:59.996029 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:53:59.996029 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:53:59.996029 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:53:59.996029 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:53:59.996029 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:53:59.996029 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:53:59.996029 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:53:59.996029 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:53:59.996029 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:53:59.996029 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:53:59.996029 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 16:53:59.996029 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:53:59.996029 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 171/348] 10.222.0.82 () {56 vars in 1114 bytes} [Thu Aug 18 16:53:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:53:59.996029 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95206a5e-a4e0-48d5-ba14-045b88646010 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:00.017360 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9251d5f-cc90-4b9f-b535-8ff4e53799b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:00.018252 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9251d5f-cc90-4b9f-b535-8ff4e53799b8 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:00.018675 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9251d5f-cc90-4b9f-b535-8ff4e53799b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:00.019118 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9251d5f-cc90-4b9f-b535-8ff4e53799b8 None admin] PATH_INFO: `/v3/users/e0de583f5ca24744930b8dea0e3a1263` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:00.020693 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:00.021236 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9251d5f-cc90-4b9f-b535-8ff4e53799b8 None admin] RBAC: Authorizing `identity:get_user(user_id=e0de583f5ca24744930b8dea0e3a1263)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:00.022892 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9251d5f-cc90-4b9f-b535-8ff4e53799b8 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:00.024990 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 178/349] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 16:53:59 2022] GET /identity/v3/users/e0de583f5ca24744930b8dea0e3a1263 => generated 293 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:00.026005 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9251d5f-cc90-4b9f-b535-8ff4e53799b8 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:00.089460 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:00.110217 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95206a5e-a4e0-48d5-ba14-045b88646010 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:00.142661 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9251d5f-cc90-4b9f-b535-8ff4e53799b8 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:00.150506 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad93a959-1f80-45fa-bcd6-a93ef8034f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:00.151400 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad93a959-1f80-45fa-bcd6-a93ef8034f5b None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:00.151821 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad93a959-1f80-45fa-bcd6-a93ef8034f5b None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:00.152230 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad93a959-1f80-45fa-bcd6-a93ef8034f5b None admin] PATH_INFO: `/v3/projects/62beed06c75a4725ae7834995533d0a0` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:00.153577 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad93a959-1f80-45fa-bcd6-a93ef8034f5b None admin] RBAC: Authorizing `identity:get_project(project_id=62beed06c75a4725ae7834995533d0a0)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:00.155311 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad93a959-1f80-45fa-bcd6-a93ef8034f5b None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:00.158051 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 172/350] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 16:53:59 2022] GET /identity/v3/projects/62beed06c75a4725ae7834995533d0a0 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:00.159160 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad93a959-1f80-45fa-bcd6-a93ef8034f5b None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:00.181612 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8b7180-0e03-402e-a2a4-2b19b80bcce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:00.182813 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8b7180-0e03-402e-a2a4-2b19b80bcce2 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:00.182813 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8b7180-0e03-402e-a2a4-2b19b80bcce2 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:00.183238 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8b7180-0e03-402e-a2a4-2b19b80bcce2 None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:00.183999 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8b7180-0e03-402e-a2a4-2b19b80bcce2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:00.184394 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8b7180-0e03-402e-a2a4-2b19b80bcce2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:00.185957 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8b7180-0e03-402e-a2a4-2b19b80bcce2 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:00.201729 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 179/351] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 16:54:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:00.202521 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8b7180-0e03-402e-a2a4-2b19b80bcce2 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:00.262936 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:00.269931 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad93a959-1f80-45fa-bcd6-a93ef8034f5b None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:00.307384 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5ae0ac-d821-4012-8f87-389114441cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:00.307998 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ae0ac-d821-4012-8f87-389114441cd5 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:00.308281 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ae0ac-d821-4012-8f87-389114441cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:00.308555 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ae0ac-d821-4012-8f87-389114441cd5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:00.309289 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5ae0ac-d821-4012-8f87-389114441cd5 None admin] RBAC: Adding query filter params (role.id=d49353712e134598b13b44d21b1e791e, user.id=e0de583f5ca24744930b8dea0e3a1263, scope.system=all) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:00.309699 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5ae0ac-d821-4012-8f87-389114441cd5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:00.311293 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5ae0ac-d821-4012-8f87-389114441cd5 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:00.325160 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 173/352] 10.222.0.82 () {56 vars in 1308 bytes} [Thu Aug 18 16:54:00 2022] GET /identity/v3/role_assignments?user.id=e0de583f5ca24744930b8dea0e3a1263&scope.system=all&role.id=d49353712e134598b13b44d21b1e791e => generated 510 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:00.326130 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5ae0ac-d821-4012-8f87-389114441cd5 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:00.361341 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8b7180-0e03-402e-a2a4-2b19b80bcce2 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:00.388060 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:00.427850 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31736073-f787-4bee-bc76-dde387f3913a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:00.428455 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31736073-f787-4bee-bc76-dde387f3913a None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:00.429050 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31736073-f787-4bee-bc76-dde387f3913a None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:00.429469 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31736073-f787-4bee-bc76-dde387f3913a None admin] PATH_INFO: `/v3/groups/f85bafc789824697bb18fdab47c54147` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:00.447676 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31736073-f787-4bee-bc76-dde387f3913a None admin] RBAC: Authorizing `identity:get_group(group_id=f85bafc789824697bb18fdab47c54147)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:00.448196 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5ae0ac-d821-4012-8f87-389114441cd5 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:00.449166 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31736073-f787-4bee-bc76-dde387f3913a None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:00.451056 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 180/353] 10.222.0.82 () {56 vars in 1180 bytes} [Thu Aug 18 16:54:00 2022] GET /identity/v3/groups/f85bafc789824697bb18fdab47c54147 => generated 226 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:00.452096 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3965f7e-08fc-4fbd-a59a-8d67009056fd None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:00.452382 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3965f7e-08fc-4fbd-a59a-8d67009056fd None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:00.452658 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3965f7e-08fc-4fbd-a59a-8d67009056fd None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:00.454958 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 181/354] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:54:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:54:00.459970 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3965f7e-08fc-4fbd-a59a-8d67009056fd None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:00.488337 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c9002f-268e-4bd2-9182-608ad79f5434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:00.489064 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c9002f-268e-4bd2-9182-608ad79f5434 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:00.489398 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c9002f-268e-4bd2-9182-608ad79f5434 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:00.489770 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c9002f-268e-4bd2-9182-608ad79f5434 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:00.511298 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:00.570644 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c9002f-268e-4bd2-9182-608ad79f5434 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:00.572285 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c9002f-268e-4bd2-9182-608ad79f5434 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:00.594613 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3965f7e-08fc-4fbd-a59a-8d67009056fd None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:00.632663 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c74ee8-cddf-48b1-a3ff-daf09ae20d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:00.633701 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c74ee8-cddf-48b1-a3ff-daf09ae20d89 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:00.634182 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c74ee8-cddf-48b1-a3ff-daf09ae20d89 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:00.635672 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c74ee8-cddf-48b1-a3ff-daf09ae20d89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:00.636636 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c74ee8-cddf-48b1-a3ff-daf09ae20d89 None admin] RBAC: Adding query filter params (group.id=f85bafc789824697bb18fdab47c54147, role.id=36d6d323d9e64e659591dbaea3081bcc, scope.project.id=62beed06c75a4725ae7834995533d0a0) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:00.636946 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c74ee8-cddf-48b1-a3ff-daf09ae20d89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:00.638428 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c74ee8-cddf-48b1-a3ff-daf09ae20d89 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:00.650388 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 182/355] 10.222.0.82 () {56 vars in 1376 bytes} [Thu Aug 18 16:54:00 2022] GET /identity/v3/role_assignments?group.id=f85bafc789824697bb18fdab47c54147&scope.project.id=62beed06c75a4725ae7834995533d0a0&role.id=36d6d323d9e64e659591dbaea3081bcc => generated 260 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:00.652119 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2a3f3-5c88-476b-9329-5de216e893f4 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:00.652408 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2a3f3-5c88-476b-9329-5de216e893f4 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:00.652684 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2a3f3-5c88-476b-9329-5de216e893f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:00.673776 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-6de2a3f3-5c88-476b-9329-5de216e893f4 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:54:00.675589 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-6de2a3f3-5c88-476b-9329-5de216e893f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:54:00.717482 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 174/356] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:54:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 16:54:00.719197 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c9002f-268e-4bd2-9182-608ad79f5434 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:00.724560 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:00.774434 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 183/357] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:54:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:54:00.775675 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de2a3f3-5c88-476b-9329-5de216e893f4 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:00.811324 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:00.886671 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c9002f-268e-4bd2-9182-608ad79f5434 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:00.943478 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8812f5ce-756d-47b5-a06e-29ef80b9dec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:00.944123 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8812f5ce-756d-47b5-a06e-29ef80b9dec4 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:00.944404 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8812f5ce-756d-47b5-a06e-29ef80b9dec4 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:00.944677 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8812f5ce-756d-47b5-a06e-29ef80b9dec4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:00.948864 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de2a3f3-5c88-476b-9329-5de216e893f4 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:00.968642 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8812f5ce-756d-47b5-a06e-29ef80b9dec4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:54:00.974868 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-8812f5ce-756d-47b5-a06e-29ef80b9dec4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 16:54:00.974868 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:00.974868 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:00.974868 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:00.974868 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:00.974868 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:00.974868 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:00.974868 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:00.974868 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:00.974868 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:00.974868 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:00.974868 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:00.974868 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:54:00.974868 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:54:00.974868 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:54:00.974868 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:54:00.975732 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:54:00.975732 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:54:00.975732 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:54:00.975732 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:54:00.975732 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:00.975732 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:00.975732 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:00.975732 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:00.975732 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:00.975732 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:00.975732 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:00.975732 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:00.975732 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:00.975732 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:00.975732 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:00.975732 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:00.976483 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:00.976483 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:00.976483 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:00.976483 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:00.976483 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:54:00.976483 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:54:00.976483 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:54:00.976483 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:54:00.976483 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:54:00.976483 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:54:00.976483 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 16:54:00.976483 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:00.976483 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 175/358] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:54:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:00.983556 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8812f5ce-756d-47b5-a06e-29ef80b9dec4 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:01.011110 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de06596-c914-473d-979d-ba8eae17f265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:01.011751 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de06596-c914-473d-979d-ba8eae17f265 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:01.012076 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de06596-c914-473d-979d-ba8eae17f265 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:01.012344 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de06596-c914-473d-979d-ba8eae17f265 None admin] PATH_INFO: `/v3/projects/62beed06c75a4725ae7834995533d0a0` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:01.014089 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de06596-c914-473d-979d-ba8eae17f265 None admin] RBAC: Authorizing `identity:get_project(project_id=62beed06c75a4725ae7834995533d0a0)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:01.016863 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de06596-c914-473d-979d-ba8eae17f265 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:01.019461 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 184/359] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 16:54:00 2022] GET /identity/v3/projects/62beed06c75a4725ae7834995533d0a0 => generated 298 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:01.020954 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c1c10-5462-42f8-8413-74e2d489afc9 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:01.021244 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c1c10-5462-42f8-8413-74e2d489afc9 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:01.023055 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c1c10-5462-42f8-8413-74e2d489afc9 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:01.023165 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 185/360] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:54:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:54:01.028081 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7c1c10-5462-42f8-8413-74e2d489afc9 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:01.106722 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:01.137495 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8812f5ce-756d-47b5-a06e-29ef80b9dec4 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:01.148048 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:01.172842 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7c1c10-5462-42f8-8413-74e2d489afc9 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:01.193051 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcd924e-7dee-4dd9-a8bc-72658a2c5a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:01.194105 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd924e-7dee-4dd9-a8bc-72658a2c5a9f None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:01.194584 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd924e-7dee-4dd9-a8bc-72658a2c5a9f None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:01.195107 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd924e-7dee-4dd9-a8bc-72658a2c5a9f None admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:01.196010 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcd924e-7dee-4dd9-a8bc-72658a2c5a9f None admin] RBAC: Adding query filter params (name=member) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:01.196511 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcd924e-7dee-4dd9-a8bc-72658a2c5a9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:01.198106 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcd924e-7dee-4dd9-a8bc-72658a2c5a9f None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:01.207501 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 176/361] 10.222.0.82 () {56 vars in 1101 bytes} [Thu Aug 18 16:54:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:54:01.209024 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a881693-4de4-44f4-9123-7ab091771836 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:01.209530 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a881693-4de4-44f4-9123-7ab091771836 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:01.210108 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a881693-4de4-44f4-9123-7ab091771836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:01.216994 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34193a5-ee09-4d38-a852-69197356af8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:01.217892 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34193a5-ee09-4d38-a852-69197356af8b None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:01.218174 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34193a5-ee09-4d38-a852-69197356af8b None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:01.219135 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34193a5-ee09-4d38-a852-69197356af8b None admin] PATH_INFO: `/v3/groups/f85bafc789824697bb18fdab47c54147` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:01.220805 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34193a5-ee09-4d38-a852-69197356af8b None admin] RBAC: Authorizing `identity:get_group(group_id=f85bafc789824697bb18fdab47c54147)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:01.222249 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34193a5-ee09-4d38-a852-69197356af8b None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:01.224197 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 186/362] 10.222.0.82 () {56 vars in 1180 bytes} [Thu Aug 18 16:54:01 2022] GET /identity/v3/groups/f85bafc789824697bb18fdab47c54147 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:01.224830 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34193a5-ee09-4d38-a852-69197356af8b None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:01.238936 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-9a881693-4de4-44f4-9123-7ab091771836 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:54:01.240253 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-9a881693-4de4-44f4-9123-7ab091771836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:54:01.321301 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 177/363] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:54:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:54:01.326923 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a881693-4de4-44f4-9123-7ab091771836 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:01.344694 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34193a5-ee09-4d38-a852-69197356af8b None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:01.359897 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:01.390020 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03bc8c1-9951-43dc-812a-c52742b89a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:01.391055 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03bc8c1-9951-43dc-812a-c52742b89a6e None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:01.391395 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03bc8c1-9951-43dc-812a-c52742b89a6e None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:01.391667 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03bc8c1-9951-43dc-812a-c52742b89a6e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:01.403936 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03bc8c1-9951-43dc-812a-c52742b89a6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:54:01.409564 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-f03bc8c1-9951-43dc-812a-c52742b89a6e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 16:54:01.409564 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:01.409564 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:01.409564 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:01.409564 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:01.409564 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:01.409564 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:01.409564 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:01.409564 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:01.409564 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:01.409564 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:01.409564 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:01.409564 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:54:01.409564 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:54:01.409564 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:54:01.409564 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:54:01.410423 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:54:01.410423 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:54:01.410423 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:54:01.410423 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:54:01.410423 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:01.410423 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:01.410423 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:01.410423 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:01.410423 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:01.410423 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:01.410423 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:01.410423 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:01.410423 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:01.410423 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:01.410423 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:01.410423 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:01.411276 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:01.411276 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:01.411276 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:01.411276 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:01.411276 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:54:01.411276 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:54:01.411276 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:54:01.411276 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:54:01.411276 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:54:01.411276 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:54:01.411276 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 16:54:01.411276 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:01.411276 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 187/364] 10.222.0.82 () {56 vars in 1114 bytes} [Thu Aug 18 16:54:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:01.411947 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03bc8c1-9951-43dc-812a-c52742b89a6e None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:01.446765 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:01.453441 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a881693-4de4-44f4-9123-7ab091771836 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:01.505623 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218ffd1c-e813-40e4-a4a3-22062cfacaf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:01.506373 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218ffd1c-e813-40e4-a4a3-22062cfacaf2 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:01.506755 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218ffd1c-e813-40e4-a4a3-22062cfacaf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:01.507191 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218ffd1c-e813-40e4-a4a3-22062cfacaf2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:01.549284 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03bc8c1-9951-43dc-812a-c52742b89a6e None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:01.588229 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218ffd1c-e813-40e4-a4a3-22062cfacaf2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:01.589709 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218ffd1c-e813-40e4-a4a3-22062cfacaf2 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:01.595044 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be0a4d1-70a8-4c2a-80e0-ecc9e55f4aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:01.595654 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be0a4d1-70a8-4c2a-80e0-ecc9e55f4aa3 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:01.595931 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be0a4d1-70a8-4c2a-80e0-ecc9e55f4aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:01.596199 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be0a4d1-70a8-4c2a-80e0-ecc9e55f4aa3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:01.596923 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be0a4d1-70a8-4c2a-80e0-ecc9e55f4aa3 None admin] RBAC: Adding query filter params (group.id=f85bafc789824697bb18fdab47c54147, role.id=f41f6f29d9de4b1985c8c8c14f002cf2, scope.project.id=62beed06c75a4725ae7834995533d0a0) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:01.597227 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be0a4d1-70a8-4c2a-80e0-ecc9e55f4aa3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:01.598663 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be0a4d1-70a8-4c2a-80e0-ecc9e55f4aa3 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:01.606083 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 188/365] 10.222.0.82 () {56 vars in 1376 bytes} [Thu Aug 18 16:54:01 2022] GET /identity/v3/role_assignments?group.id=f85bafc789824697bb18fdab47c54147&scope.project.id=62beed06c75a4725ae7834995533d0a0&role.id=f41f6f29d9de4b1985c8c8c14f002cf2 => generated 260 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:01.607279 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be0a4d1-70a8-4c2a-80e0-ecc9e55f4aa3 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:01.658126 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:01.740805 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:01.749036 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 178/366] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:54:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 16:54:01.750263 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218ffd1c-e813-40e4-a4a3-22062cfacaf2 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:01.766378 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be0a4d1-70a8-4c2a-80e0-ecc9e55f4aa3 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:01.822250 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff81def5-f851-4d41-a724-86dd0bfa441c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:01.823258 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81def5-f851-4d41-a724-86dd0bfa441c None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:01.823778 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81def5-f851-4d41-a724-86dd0bfa441c None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:01.824255 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81def5-f851-4d41-a724-86dd0bfa441c None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:01.824925 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff81def5-f851-4d41-a724-86dd0bfa441c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:01.825235 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff81def5-f851-4d41-a724-86dd0bfa441c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:01.826642 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff81def5-f851-4d41-a724-86dd0bfa441c None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:01.836702 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 189/367] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 16:54:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:01.837338 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff81def5-f851-4d41-a724-86dd0bfa441c None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:01.897045 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218ffd1c-e813-40e4-a4a3-22062cfacaf2 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:01.940739 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239a314d-a2da-4eca-838c-4a22691ad456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:01.941406 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a314d-a2da-4eca-838c-4a22691ad456 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:01.941687 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a314d-a2da-4eca-838c-4a22691ad456 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:01.941956 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a314d-a2da-4eca-838c-4a22691ad456 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:01.956228 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239a314d-a2da-4eca-838c-4a22691ad456 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:54:01.959959 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:01.962933 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-239a314d-a2da-4eca-838c-4a22691ad456 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 16:54:01.962933 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:01.962933 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:01.962933 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:01.962933 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:01.962933 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:01.962933 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:01.962933 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:01.962933 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:01.962933 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:01.962933 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:01.962933 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:01.962933 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:54:01.962933 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:54:01.962933 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:54:01.962933 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:54:01.963769 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:54:01.963769 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:54:01.963769 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:54:01.963769 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:54:01.963769 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:01.963769 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:01.963769 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:01.963769 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:01.963769 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:01.963769 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:01.963769 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:01.963769 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:01.963769 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:01.963769 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:01.963769 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:01.963769 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:01.964519 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:01.964519 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:01.964519 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:01.964519 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:01.964519 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:54:01.964519 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:54:01.964519 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:54:01.964519 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:54:01.964519 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:54:01.964519 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:54:01.964519 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 16:54:01.964519 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:01.964519 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 179/368] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 16:54:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:01.965737 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239a314d-a2da-4eca-838c-4a22691ad456 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:01.973015 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff81def5-f851-4d41-a724-86dd0bfa441c None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:02.013878 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d4264f-f82f-481a-b919-75218b5c31b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:02.015007 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d4264f-f82f-481a-b919-75218b5c31b8 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:02.015735 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d4264f-f82f-481a-b919-75218b5c31b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:02.016225 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d4264f-f82f-481a-b919-75218b5c31b8 None admin] PATH_INFO: `/v3/projects/d5b629fedcc945bf8c55bda4ceedc4ae` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:02.017745 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d4264f-f82f-481a-b919-75218b5c31b8 None admin] RBAC: Authorizing `identity:get_project(project_id=d5b629fedcc945bf8c55bda4ceedc4ae)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:02.019595 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d4264f-f82f-481a-b919-75218b5c31b8 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:02.021840 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 190/369] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 16:54:01 2022] GET /identity/v3/projects/d5b629fedcc945bf8c55bda4ceedc4ae => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:02.022855 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d4264f-f82f-481a-b919-75218b5c31b8 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:02.042123 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:02.095859 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239a314d-a2da-4eca-838c-4a22691ad456 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:02.139577 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1cbb8d-39fe-42fd-9eba-1e5c9055d4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:02.140728 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1cbb8d-39fe-42fd-9eba-1e5c9055d4d2 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:02.141020 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1cbb8d-39fe-42fd-9eba-1e5c9055d4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:02.141292 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1cbb8d-39fe-42fd-9eba-1e5c9055d4d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:02.149774 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d4264f-f82f-481a-b919-75218b5c31b8 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:02.191141 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b921c8-c2d8-4b20-a587-58d68d52de3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:02.192276 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b921c8-c2d8-4b20-a587-58d68d52de3f None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:02.192858 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b921c8-c2d8-4b20-a587-58d68d52de3f None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:02.193319 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b921c8-c2d8-4b20-a587-58d68d52de3f None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:02.194202 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b921c8-c2d8-4b20-a587-58d68d52de3f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:02.194791 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b921c8-c2d8-4b20-a587-58d68d52de3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:02.196430 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b921c8-c2d8-4b20-a587-58d68d52de3f None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:02.207647 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 191/370] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:54:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:02.208872 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b921c8-c2d8-4b20-a587-58d68d52de3f None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:02.226133 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1cbb8d-39fe-42fd-9eba-1e5c9055d4d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:02.227791 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1cbb8d-39fe-42fd-9eba-1e5c9055d4d2 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:02.240747 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:02.296504 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:02.329377 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b921c8-c2d8-4b20-a587-58d68d52de3f None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:02.345399 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 180/371] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:54:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 16:54:02.346376 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1cbb8d-39fe-42fd-9eba-1e5c9055d4d2 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:02.370184 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c175975d-6daf-403a-851f-323f2ec7f0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:02.371290 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c175975d-6daf-403a-851f-323f2ec7f0e6 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:02.371765 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c175975d-6daf-403a-851f-323f2ec7f0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:02.372298 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c175975d-6daf-403a-851f-323f2ec7f0e6 None admin] PATH_INFO: `/v3/groups/f85bafc789824697bb18fdab47c54147` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:02.373763 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c175975d-6daf-403a-851f-323f2ec7f0e6 None admin] RBAC: Authorizing `identity:get_group(group_id=f85bafc789824697bb18fdab47c54147)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:02.376360 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c175975d-6daf-403a-851f-323f2ec7f0e6 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:02.379361 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 192/372] 10.222.0.82 () {56 vars in 1180 bytes} [Thu Aug 18 16:54:02 2022] GET /identity/v3/groups/f85bafc789824697bb18fdab47c54147 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:02.379361 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c175975d-6daf-403a-851f-323f2ec7f0e6 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:02.457493 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1cbb8d-39fe-42fd-9eba-1e5c9055d4d2 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:02.488384 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:02.495014 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6e74c4-ed6e-4379-a5b7-ff3cb7eb9086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:02.495645 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e74c4-ed6e-4379-a5b7-ff3cb7eb9086 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:02.495975 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e74c4-ed6e-4379-a5b7-ff3cb7eb9086 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:02.496201 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e74c4-ed6e-4379-a5b7-ff3cb7eb9086 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:02.501406 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c175975d-6daf-403a-851f-323f2ec7f0e6 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:02.525957 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:02.541837 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b298691-7a1c-41d5-8ec4-b101c2b713af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:02.542805 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b298691-7a1c-41d5-8ec4-b101c2b713af None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:02.543304 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b298691-7a1c-41d5-8ec4-b101c2b713af None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:02.543827 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b298691-7a1c-41d5-8ec4-b101c2b713af None admin] PATH_INFO: `/v3/projects/d5b629fedcc945bf8c55bda4ceedc4ae` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:02.545184 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b298691-7a1c-41d5-8ec4-b101c2b713af None admin] RBAC: Authorizing `identity:get_project(project_id=d5b629fedcc945bf8c55bda4ceedc4ae)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:02.547276 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b298691-7a1c-41d5-8ec4-b101c2b713af None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:02.549454 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 193/373] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 16:54:02 2022] GET /identity/v3/projects/d5b629fedcc945bf8c55bda4ceedc4ae => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:02.550496 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b298691-7a1c-41d5-8ec4-b101c2b713af None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:02.568728 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6e74c4-ed6e-4379-a5b7-ff3cb7eb9086 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:02.570236 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6e74c4-ed6e-4379-a5b7-ff3cb7eb9086 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:02.665660 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b298691-7a1c-41d5-8ec4-b101c2b713af None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:02.675601 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 181/374] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:54:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 16:54:02.676538 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6e74c4-ed6e-4379-a5b7-ff3cb7eb9086 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:02.706420 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f903d8d5-eea1-4ced-baf5-abcc37712840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:02.707493 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903d8d5-eea1-4ced-baf5-abcc37712840 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:02.707985 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903d8d5-eea1-4ced-baf5-abcc37712840 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:02.708475 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903d8d5-eea1-4ced-baf5-abcc37712840 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:02.709383 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f903d8d5-eea1-4ced-baf5-abcc37712840 None admin] RBAC: Adding query filter params (group.id=f85bafc789824697bb18fdab47c54147, role.id=36d6d323d9e64e659591dbaea3081bcc, scope.project.id=d5b629fedcc945bf8c55bda4ceedc4ae) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:02.709880 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f903d8d5-eea1-4ced-baf5-abcc37712840 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:02.711633 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f903d8d5-eea1-4ced-baf5-abcc37712840 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:02.718158 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 194/375] 10.222.0.82 () {56 vars in 1376 bytes} [Thu Aug 18 16:54:02 2022] GET /identity/v3/role_assignments?group.id=f85bafc789824697bb18fdab47c54147&scope.project.id=d5b629fedcc945bf8c55bda4ceedc4ae&role.id=36d6d323d9e64e659591dbaea3081bcc => generated 260 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:02.720164 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f903d8d5-eea1-4ced-baf5-abcc37712840 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:02.749842 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:02.754893 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:02.822678 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6e74c4-ed6e-4379-a5b7-ff3cb7eb9086 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:02.840150 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f903d8d5-eea1-4ced-baf5-abcc37712840 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:02.872074 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e23f1d-8270-45b9-b045-440868c35144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:02.872740 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e23f1d-8270-45b9-b045-440868c35144 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:02.873026 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e23f1d-8270-45b9-b045-440868c35144 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:02.873304 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e23f1d-8270-45b9-b045-440868c35144 None admin] PATH_INFO: `/v3/groups/f85bafc789824697bb18fdab47c54147` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:02.874562 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e23f1d-8270-45b9-b045-440868c35144 None admin] RBAC: Authorizing `identity:get_group(group_id=f85bafc789824697bb18fdab47c54147)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:02.876107 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e23f1d-8270-45b9-b045-440868c35144 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:02.880680 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 182/376] 10.222.0.82 () {56 vars in 1180 bytes} [Thu Aug 18 16:54:02 2022] GET /identity/v3/groups/f85bafc789824697bb18fdab47c54147 => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:02.881685 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8ee96b-b939-4ab2-a214-e10b0304ea71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:02.882117 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7748c-5bb5-45cb-9e29-c784ca7167c2 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:02.882341 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ee96b-b939-4ab2-a214-e10b0304ea71 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:02.882440 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7748c-5bb5-45cb-9e29-c784ca7167c2 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:02.882617 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ee96b-b939-4ab2-a214-e10b0304ea71 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:02.882712 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7748c-5bb5-45cb-9e29-c784ca7167c2 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:02.882886 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ee96b-b939-4ab2-a214-e10b0304ea71 None admin] PATH_INFO: `/v3/projects/77b6b270950a499cb6b0da7c630c03b6` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:02.884280 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8ee96b-b939-4ab2-a214-e10b0304ea71 None admin] RBAC: Authorizing `identity:get_project(project_id=77b6b270950a499cb6b0da7c630c03b6)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:02.884392 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 183/377] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:54:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:54:02.886794 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8ee96b-b939-4ab2-a214-e10b0304ea71 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:02.889112 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 195/378] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 16:54:02 2022] GET /identity/v3/projects/77b6b270950a499cb6b0da7c630c03b6 => generated 344 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:02.889254 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d7748c-5bb5-45cb-9e29-c784ca7167c2 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:02.892275 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff14ffe-b7a7-4c10-8f91-e38605a8afeb None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:02.892630 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff14ffe-b7a7-4c10-8f91-e38605a8afeb None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:02.893192 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff14ffe-b7a7-4c10-8f91-e38605a8afeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:02.916032 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-6ff14ffe-b7a7-4c10-8f91-e38605a8afeb None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:54:02.917324 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-6ff14ffe-b7a7-4c10-8f91-e38605a8afeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:54:03.005993 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 196/379] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:54:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 18 16:54:03.006851 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff14ffe-b7a7-4c10-8f91-e38605a8afeb None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:03.014733 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d7748c-5bb5-45cb-9e29-c784ca7167c2 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:03.033195 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:03.044939 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:03.055248 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c210208-398d-43e5-b274-e9c426e5c3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:03.056259 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c210208-398d-43e5-b274-e9c426e5c3e4 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:03.056688 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c210208-398d-43e5-b274-e9c426e5c3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:03.057105 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c210208-398d-43e5-b274-e9c426e5c3e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:03.058085 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c210208-398d-43e5-b274-e9c426e5c3e4 None admin] RBAC: Adding query filter params (group.id=f85bafc789824697bb18fdab47c54147, role.id=f41f6f29d9de4b1985c8c8c14f002cf2, scope.project.id=d5b629fedcc945bf8c55bda4ceedc4ae) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:03.058539 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c210208-398d-43e5-b274-e9c426e5c3e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:03.060214 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c210208-398d-43e5-b274-e9c426e5c3e4 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:03.067109 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 184/380] 10.222.0.82 () {56 vars in 1376 bytes} [Thu Aug 18 16:54:02 2022] GET /identity/v3/role_assignments?group.id=f85bafc789824697bb18fdab47c54147&scope.project.id=d5b629fedcc945bf8c55bda4ceedc4ae&role.id=f41f6f29d9de4b1985c8c8c14f002cf2 => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:03.119964 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff14ffe-b7a7-4c10-8f91-e38605a8afeb None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:03.159408 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96fc2d4-2aab-4d67-8c70-e59b262ea5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:03.160065 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96fc2d4-2aab-4d67-8c70-e59b262ea5a4 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:03.160341 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96fc2d4-2aab-4d67-8c70-e59b262ea5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:03.160614 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96fc2d4-2aab-4d67-8c70-e59b262ea5a4 None admin] PATH_INFO: `/v3/groups/a8bda1d009254ec6a9e5260b54e0f3d6` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:03.166033 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96fc2d4-2aab-4d67-8c70-e59b262ea5a4 None admin] RBAC: Authorizing `identity:get_group(group_id=a8bda1d009254ec6a9e5260b54e0f3d6)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:03.167525 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96fc2d4-2aab-4d67-8c70-e59b262ea5a4 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:03.169528 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 197/381] 10.222.0.82 () {56 vars in 1180 bytes} [Thu Aug 18 16:54:03 2022] GET /identity/v3/groups/a8bda1d009254ec6a9e5260b54e0f3d6 => generated 229 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:03.175116 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c210208-398d-43e5-b274-e9c426e5c3e4 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:03.302794 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c210208-398d-43e5-b274-e9c426e5c3e4 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:03.339781 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d810e731-a36b-4476-81f2-f1d4e30e50b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:03.340645 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d810e731-a36b-4476-81f2-f1d4e30e50b7 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:03.341114 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d810e731-a36b-4476-81f2-f1d4e30e50b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:03.343770 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d810e731-a36b-4476-81f2-f1d4e30e50b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:03.343770 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d810e731-a36b-4476-81f2-f1d4e30e50b7 None admin] RBAC: Adding query filter params (group.id=a8bda1d009254ec6a9e5260b54e0f3d6, role.id=2c6966cfb10f4d72b0242e9fb5e13b17, scope.project.id=77b6b270950a499cb6b0da7c630c03b6) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:03.343770 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d810e731-a36b-4476-81f2-f1d4e30e50b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:03.345514 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d810e731-a36b-4476-81f2-f1d4e30e50b7 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:03.352395 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 185/382] 10.222.0.82 () {56 vars in 1376 bytes} [Thu Aug 18 16:54:03 2022] GET /identity/v3/role_assignments?group.id=a8bda1d009254ec6a9e5260b54e0f3d6&scope.project.id=77b6b270950a499cb6b0da7c630c03b6&role.id=2c6966cfb10f4d72b0242e9fb5e13b17 => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:03.403897 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96fc2d4-2aab-4d67-8c70-e59b262ea5a4 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:03.510957 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:03.532401 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96fc2d4-2aab-4d67-8c70-e59b262ea5a4 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:03.575442 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68afd0c-79ef-45a5-85c8-0d2e62f48cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:03.576077 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68afd0c-79ef-45a5-85c8-0d2e62f48cd2 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:03.576359 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68afd0c-79ef-45a5-85c8-0d2e62f48cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:03.576626 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68afd0c-79ef-45a5-85c8-0d2e62f48cd2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:03.587398 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68afd0c-79ef-45a5-85c8-0d2e62f48cd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:54:03.592320 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-d68afd0c-79ef-45a5-85c8-0d2e62f48cd2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 16:54:03.592320 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:03.592320 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:03.592320 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:03.592320 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:03.592320 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:03.592320 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:03.592320 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:03.592320 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:03.592320 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:03.592320 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:03.592320 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:03.592320 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:54:03.592320 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:54:03.592320 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:54:03.592320 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:54:03.593582 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:54:03.593582 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:54:03.593582 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:54:03.593582 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:54:03.593582 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:03.593582 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:03.593582 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:03.593582 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:03.593582 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:03.593582 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:03.593582 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:03.593582 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:03.593582 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:03.593582 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:03.593582 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:03.593582 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:03.595117 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:03.595117 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:03.595117 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:03.595117 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:03.595117 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:54:03.595117 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:54:03.595117 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:54:03.595117 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:54:03.595117 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:54:03.595117 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:54:03.595117 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 16:54:03.595117 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:03.595117 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 198/383] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:54:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:03.599367 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d810e731-a36b-4476-81f2-f1d4e30e50b7 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:03.685163 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:03.741374 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d810e731-a36b-4476-81f2-f1d4e30e50b7 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:03.785911 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e43a3f-abc1-4287-aa83-7e417ab6541d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:03.785911 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e43a3f-abc1-4287-aa83-7e417ab6541d None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:03.786195 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e43a3f-abc1-4287-aa83-7e417ab6541d None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:03.786423 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e43a3f-abc1-4287-aa83-7e417ab6541d None admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:03.787031 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e43a3f-abc1-4287-aa83-7e417ab6541d None admin] RBAC: Adding query filter params (name=member) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:03.787351 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e43a3f-abc1-4287-aa83-7e417ab6541d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:03.788809 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e43a3f-abc1-4287-aa83-7e417ab6541d None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:03.802323 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 186/384] 10.222.0.82 () {56 vars in 1101 bytes} [Thu Aug 18 16:54:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:03.805157 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68afd0c-79ef-45a5-85c8-0d2e62f48cd2 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:03.936222 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68afd0c-79ef-45a5-85c8-0d2e62f48cd2 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:03.975794 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27a2aef-92e3-4c9f-85de-f6198da1f726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:03.976429 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a2aef-92e3-4c9f-85de-f6198da1f726 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:03.976705 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a2aef-92e3-4c9f-85de-f6198da1f726 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:03.976977 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a2aef-92e3-4c9f-85de-f6198da1f726 None admin] PATH_INFO: `/v3/groups/f85bafc789824697bb18fdab47c54147` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:03.978064 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27a2aef-92e3-4c9f-85de-f6198da1f726 None admin] RBAC: Authorizing `identity:get_group(group_id=f85bafc789824697bb18fdab47c54147)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:03.979538 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27a2aef-92e3-4c9f-85de-f6198da1f726 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:03.981250 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 199/385] 10.222.0.82 () {56 vars in 1180 bytes} [Thu Aug 18 16:54:03 2022] GET /identity/v3/groups/f85bafc789824697bb18fdab47c54147 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:03.992023 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e43a3f-abc1-4287-aa83-7e417ab6541d None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:04.120458 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e43a3f-abc1-4287-aa83-7e417ab6541d None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:04.139240 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:04.166430 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aafdfc-9668-4cb9-9d48-d2a901b77825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:04.167136 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aafdfc-9668-4cb9-9d48-d2a901b77825 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:04.167421 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aafdfc-9668-4cb9-9d48-d2a901b77825 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:04.167702 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aafdfc-9668-4cb9-9d48-d2a901b77825 None admin] PATH_INFO: `/v3/projects/62beed06c75a4725ae7834995533d0a0` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:04.168950 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aafdfc-9668-4cb9-9d48-d2a901b77825 None admin] RBAC: Authorizing `identity:get_project(project_id=62beed06c75a4725ae7834995533d0a0)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:04.170515 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aafdfc-9668-4cb9-9d48-d2a901b77825 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:04.173881 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 187/386] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 16:54:03 2022] GET /identity/v3/projects/62beed06c75a4725ae7834995533d0a0 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:04.179931 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27a2aef-92e3-4c9f-85de-f6198da1f726 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:04.202876 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:04.317106 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27a2aef-92e3-4c9f-85de-f6198da1f726 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:04.356825 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f9cca9-6984-404e-8e5a-f4b5a41922a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:04.357491 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f9cca9-6984-404e-8e5a-f4b5a41922a3 None admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:04.357769 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f9cca9-6984-404e-8e5a-f4b5a41922a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:04.358040 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f9cca9-6984-404e-8e5a-f4b5a41922a3 None admin] PATH_INFO: `/v3/projects/62beed06c75a4725ae7834995533d0a0/groups/f85bafc789824697bb18fdab47c54147/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:04.360888 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f9cca9-6984-404e-8e5a-f4b5a41922a3 None admin] RBAC: Authorizing `identity:create_grant(project_id=62beed06c75a4725ae7834995533d0a0, group_id=f85bafc789824697bb18fdab47c54147, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:04.362311 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f9cca9-6984-404e-8e5a-f4b5a41922a3 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:04.380742 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3db32b-da42-4a31-901c-6df47beef47b None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:04.381037 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3db32b-da42-4a31-901c-6df47beef47b None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:04.381305 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3db32b-da42-4a31-901c-6df47beef47b None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:04.382928 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 188/387] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:54:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:54:04.387463 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07896710-d3bf-4a33-9996-43640b4dd245 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:04.387753 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07896710-d3bf-4a33-9996-43640b4dd245 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:04.388033 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07896710-d3bf-4a33-9996-43640b4dd245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:04.417174 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-07896710-d3bf-4a33-9996-43640b4dd245 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:54:04.418378 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-07896710-d3bf-4a33-9996-43640b4dd245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:54:04.444371 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 200/388] 10.222.0.82 () {58 vars in 1442 bytes} [Thu Aug 18 16:54:04 2022] PUT /identity/v3/projects/62beed06c75a4725ae7834995533d0a0/groups/f85bafc789824697bb18fdab47c54147/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 16:54:04.507516 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 189/389] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:54:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 16:54:04.792537 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f9cca9-6984-404e-8e5a-f4b5a41922a3 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:04.884728 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:04.919407 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f9cca9-6984-404e-8e5a-f4b5a41922a3 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:04.955238 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff186f4e-b5c2-4b20-8a48-d06b3d067e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:04.955880 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff186f4e-b5c2-4b20-8a48-d06b3d067e09 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:04.956156 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff186f4e-b5c2-4b20-8a48-d06b3d067e09 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:04.956427 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff186f4e-b5c2-4b20-8a48-d06b3d067e09 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:04.979849 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff186f4e-b5c2-4b20-8a48-d06b3d067e09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:54:04.984946 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-ff186f4e-b5c2-4b20-8a48-d06b3d067e09 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 16:54:04.984946 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:04.984946 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:04.984946 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:04.984946 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:04.984946 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:04.984946 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:04.984946 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:04.984946 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:04.984946 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:04.984946 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:04.984946 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:04.984946 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:54:04.984946 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:54:04.984946 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:54:04.984946 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:54:04.985840 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:54:04.985840 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:54:04.985840 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:54:04.985840 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:54:04.985840 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:04.985840 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:04.985840 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:04.985840 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:04.985840 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:04.985840 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:04.985840 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:04.985840 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:04.985840 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:04.985840 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:04.985840 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:04.985840 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:04.987125 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:04.987125 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:04.987125 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:04.987125 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:04.987125 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:54:04.987125 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:54:04.987125 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:54:04.987125 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:54:04.987125 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:54:04.987125 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:54:04.987125 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 16:54:04.987125 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:04.987125 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 201/390] 10.222.0.82 () {56 vars in 1114 bytes} [Thu Aug 18 16:54:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:04.992137 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07896710-d3bf-4a33-9996-43640b4dd245 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:05.103504 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07896710-d3bf-4a33-9996-43640b4dd245 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:05.123253 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:05.145006 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6ee21b-1722-4517-bc5f-6d63fea6e7c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:05.145639 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ee21b-1722-4517-bc5f-6d63fea6e7c8 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:05.145922 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ee21b-1722-4517-bc5f-6d63fea6e7c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:05.146197 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ee21b-1722-4517-bc5f-6d63fea6e7c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:05.146874 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6ee21b-1722-4517-bc5f-6d63fea6e7c8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:05.147203 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6ee21b-1722-4517-bc5f-6d63fea6e7c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:05.148600 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6ee21b-1722-4517-bc5f-6d63fea6e7c8 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:05.157724 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 190/391] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 16:54:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:05.163134 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff186f4e-b5c2-4b20-8a48-d06b3d067e09 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:05.272106 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff186f4e-b5c2-4b20-8a48-d06b3d067e09 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:05.313613 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ac488c-679a-44ef-9029-8a8e688df11b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:05.314262 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac488c-679a-44ef-9029-8a8e688df11b None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:05.314542 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac488c-679a-44ef-9029-8a8e688df11b None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:05.314817 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac488c-679a-44ef-9029-8a8e688df11b None admin] PATH_INFO: `/v3/groups/f85bafc789824697bb18fdab47c54147` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:05.319561 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ac488c-679a-44ef-9029-8a8e688df11b None admin] RBAC: Authorizing `identity:get_group(group_id=f85bafc789824697bb18fdab47c54147)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:05.321027 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ac488c-679a-44ef-9029-8a8e688df11b None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:05.322860 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 202/392] 10.222.0.82 () {56 vars in 1180 bytes} [Thu Aug 18 16:54:05 2022] GET /identity/v3/groups/f85bafc789824697bb18fdab47c54147 => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:05.328471 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6ee21b-1722-4517-bc5f-6d63fea6e7c8 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:05.465225 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6ee21b-1722-4517-bc5f-6d63fea6e7c8 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:05.477434 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e609a58-209c-4708-99c2-f4bcca0850e2 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:05.477736 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e609a58-209c-4708-99c2-f4bcca0850e2 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:05.477982 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e609a58-209c-4708-99c2-f4bcca0850e2 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:05.479621 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 203/393] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:54:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:54:05.480677 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125f4ea8-1129-4569-bd90-d1dbf9fd4bc8 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:05.480956 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125f4ea8-1129-4569-bd90-d1dbf9fd4bc8 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:05.481228 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125f4ea8-1129-4569-bd90-d1dbf9fd4bc8 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:05.482397 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 204/394] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:54:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:54:05.489162 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfaa4ed-0683-420a-ad3f-4bc3c5130545 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:05.489453 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfaa4ed-0683-420a-ad3f-4bc3c5130545 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:05.489731 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfaa4ed-0683-420a-ad3f-4bc3c5130545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:05.497051 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:05.512272 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-bdfaa4ed-0683-420a-ad3f-4bc3c5130545 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:54:05.513543 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-bdfaa4ed-0683-420a-ad3f-4bc3c5130545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:54:05.515521 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477494cd-b06c-47ec-ac86-f712ec2ba526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:05.516175 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477494cd-b06c-47ec-ac86-f712ec2ba526 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:05.516457 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477494cd-b06c-47ec-ac86-f712ec2ba526 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:05.516735 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477494cd-b06c-47ec-ac86-f712ec2ba526 None admin] PATH_INFO: `/v3/projects/62beed06c75a4725ae7834995533d0a0` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:05.517880 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477494cd-b06c-47ec-ac86-f712ec2ba526 None admin] RBAC: Authorizing `identity:get_project(project_id=62beed06c75a4725ae7834995533d0a0)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:05.519430 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477494cd-b06c-47ec-ac86-f712ec2ba526 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:05.521365 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 191/395] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 16:54:05 2022] GET /identity/v3/projects/62beed06c75a4725ae7834995533d0a0 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:05.522499 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239081c0-90b7-4b2c-8f7b-0fada5e0dac1 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:05.522786 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239081c0-90b7-4b2c-8f7b-0fada5e0dac1 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:05.527650 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239081c0-90b7-4b2c-8f7b-0fada5e0dac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:05.549313 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-239081c0-90b7-4b2c-8f7b-0fada5e0dac1 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:54:05.551135 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-239081c0-90b7-4b2c-8f7b-0fada5e0dac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:54:05.590987 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 205/396] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:54:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 16:54:05.592431 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfaa4ed-0683-420a-ad3f-4bc3c5130545 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:05.607013 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:05.658031 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 192/397] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:54:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:54:05.748400 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfaa4ed-0683-420a-ad3f-4bc3c5130545 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:05.781117 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:05.792577 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4647ac-f179-4e8a-9e8c-6f4f1fdbf1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:05.793260 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4647ac-f179-4e8a-9e8c-6f4f1fdbf1e6 None admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:05.793544 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4647ac-f179-4e8a-9e8c-6f4f1fdbf1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:05.793817 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4647ac-f179-4e8a-9e8c-6f4f1fdbf1e6 None admin] PATH_INFO: `/v3/projects/62beed06c75a4725ae7834995533d0a0/groups/f85bafc789824697bb18fdab47c54147/roles/f41f6f29d9de4b1985c8c8c14f002cf2` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:05.796009 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4647ac-f179-4e8a-9e8c-6f4f1fdbf1e6 None admin] RBAC: Authorizing `identity:create_grant(project_id=62beed06c75a4725ae7834995533d0a0, group_id=f85bafc789824697bb18fdab47c54147, role_id=f41f6f29d9de4b1985c8c8c14f002cf2)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:05.797976 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4647ac-f179-4e8a-9e8c-6f4f1fdbf1e6 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:05.914622 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 206/398] 10.222.0.82 () {58 vars in 1442 bytes} [Thu Aug 18 16:54:05 2022] PUT /identity/v3/projects/62beed06c75a4725ae7834995533d0a0/groups/f85bafc789824697bb18fdab47c54147/roles/f41f6f29d9de4b1985c8c8c14f002cf2 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 16:54:05.962601 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239081c0-90b7-4b2c-8f7b-0fada5e0dac1 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:06.013732 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4647ac-f179-4e8a-9e8c-6f4f1fdbf1e6 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:06.104358 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239081c0-90b7-4b2c-8f7b-0fada5e0dac1 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:06.155653 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731fdd2c-e7eb-49cc-8131-2808f4da6889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:06.157030 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731fdd2c-e7eb-49cc-8131-2808f4da6889 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:06.157312 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731fdd2c-e7eb-49cc-8131-2808f4da6889 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:06.157587 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731fdd2c-e7eb-49cc-8131-2808f4da6889 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:06.177715 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731fdd2c-e7eb-49cc-8131-2808f4da6889 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:54:06.183613 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-731fdd2c-e7eb-49cc-8131-2808f4da6889 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 16:54:06.183613 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:06.183613 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:06.183613 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:06.183613 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:06.183613 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:06.183613 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:06.183613 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:06.183613 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:06.183613 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:06.183613 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:06.183613 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:06.183613 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:54:06.183613 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:54:06.183613 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:54:06.183613 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:54:06.184561 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:54:06.184561 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:54:06.184561 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:54:06.184561 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:54:06.184561 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:06.184561 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:06.184561 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:06.184561 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:06.184561 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:06.184561 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:06.184561 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:06.184561 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:06.184561 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:06.184561 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:06.184561 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:06.184561 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:06.185359 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:06.185359 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:06.185359 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:06.185359 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:06.185359 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:54:06.185359 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:54:06.185359 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:54:06.185359 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:54:06.185359 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:54:06.185359 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:54:06.185359 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 16:54:06.185359 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:06.185359 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 193/399] 10.222.0.82 () {56 vars in 1114 bytes} [Thu Aug 18 16:54:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:06.187843 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67900e-b1d1-4b81-acd7-2975cb8c8597 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:06.188389 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67900e-b1d1-4b81-acd7-2975cb8c8597 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:06.188883 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67900e-b1d1-4b81-acd7-2975cb8c8597 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:06.191025 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 194/400] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:54:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:54:06.193681 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f67900e-b1d1-4b81-acd7-2975cb8c8597 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:06.199189 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4647ac-f179-4e8a-9e8c-6f4f1fdbf1e6 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:06.257860 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:06.258058 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f163d407-318c-49ea-94f0-e9f6167ff544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:06.258887 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163d407-318c-49ea-94f0-e9f6167ff544 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:06.260096 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163d407-318c-49ea-94f0-e9f6167ff544 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:06.260096 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163d407-318c-49ea-94f0-e9f6167ff544 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:06.279936 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f163d407-318c-49ea-94f0-e9f6167ff544 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:54:06.285449 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-f163d407-318c-49ea-94f0-e9f6167ff544 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 16:54:06.285449 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:06.285449 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:06.285449 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:06.285449 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:06.285449 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:06.285449 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:06.285449 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:06.285449 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:06.285449 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:06.285449 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:06.285449 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:06.285449 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:54:06.285449 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:54:06.285449 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:54:06.285449 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:54:06.286330 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:54:06.286330 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:54:06.286330 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:54:06.286330 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:54:06.286330 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:06.286330 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:06.286330 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:06.286330 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:06.286330 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:06.286330 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:06.286330 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:06.286330 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:06.286330 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:06.286330 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:06.286330 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:06.286330 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:06.287120 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:06.287120 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:06.287120 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:06.287120 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:06.287120 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:54:06.287120 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:54:06.287120 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:54:06.287120 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:54:06.287120 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:54:06.287120 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:54:06.287120 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 16:54:06.287120 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:06.287120 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 207/401] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:54:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:06.292266 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26161dc-5f08-48ff-91e7-e62006739e1b None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:06.292813 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26161dc-5f08-48ff-91e7-e62006739e1b None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:06.292917 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26161dc-5f08-48ff-91e7-e62006739e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:06.316423 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-e26161dc-5f08-48ff-91e7-e62006739e1b None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:54:06.318209 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-e26161dc-5f08-48ff-91e7-e62006739e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:54:06.350836 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f67900e-b1d1-4b81-acd7-2975cb8c8597 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:06.371258 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:06.390826 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeb62a5-c6da-4ddf-99ba-3b5a10190d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:06.391645 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb62a5-c6da-4ddf-99ba-3b5a10190d7d None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:06.392012 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb62a5-c6da-4ddf-99ba-3b5a10190d7d None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:06.392401 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb62a5-c6da-4ddf-99ba-3b5a10190d7d None admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:06.393208 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeb62a5-c6da-4ddf-99ba-3b5a10190d7d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:06.393621 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeb62a5-c6da-4ddf-99ba-3b5a10190d7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:06.394588 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 208/402] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:54:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:54:06.395261 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeb62a5-c6da-4ddf-99ba-3b5a10190d7d None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:06.395872 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26161dc-5f08-48ff-91e7-e62006739e1b None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:06.404624 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 195/403] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 16:54:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:54:06.410536 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeb62a5-c6da-4ddf-99ba-3b5a10190d7d None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:06.538374 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:06.546999 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeb62a5-c6da-4ddf-99ba-3b5a10190d7d None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:06.564653 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26161dc-5f08-48ff-91e7-e62006739e1b None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:06.592829 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b096fc80-87ff-4d4a-96d8-e2cf3a3227b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:06.593655 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b096fc80-87ff-4d4a-96d8-e2cf3a3227b9 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:06.594549 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b096fc80-87ff-4d4a-96d8-e2cf3a3227b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:06.594953 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b096fc80-87ff-4d4a-96d8-e2cf3a3227b9 None admin] PATH_INFO: `/v3/groups/f85bafc789824697bb18fdab47c54147` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:06.597753 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b096fc80-87ff-4d4a-96d8-e2cf3a3227b9 None admin] RBAC: Authorizing `identity:get_group(group_id=f85bafc789824697bb18fdab47c54147)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:06.599960 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b096fc80-87ff-4d4a-96d8-e2cf3a3227b9 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:06.602629 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 196/404] 10.222.0.82 () {56 vars in 1180 bytes} [Thu Aug 18 16:54:06 2022] GET /identity/v3/groups/f85bafc789824697bb18fdab47c54147 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:54:06.607650 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b096fc80-87ff-4d4a-96d8-e2cf3a3227b9 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:06.610816 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb31dd8-43d9-4c62-80f0-3ef20d9717b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:06.612257 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb31dd8-43d9-4c62-80f0-3ef20d9717b0 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:06.612538 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb31dd8-43d9-4c62-80f0-3ef20d9717b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:06.612862 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb31dd8-43d9-4c62-80f0-3ef20d9717b0 None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:06.613543 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb31dd8-43d9-4c62-80f0-3ef20d9717b0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:06.614840 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb31dd8-43d9-4c62-80f0-3ef20d9717b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:06.616257 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb31dd8-43d9-4c62-80f0-3ef20d9717b0 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:06.627800 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 209/405] 10.222.0.82 () {56 vars in 1101 bytes} [Thu Aug 18 16:54:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:06.634788 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb31dd8-43d9-4c62-80f0-3ef20d9717b0 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:06.711315 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:06.757156 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b096fc80-87ff-4d4a-96d8-e2cf3a3227b9 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:06.774469 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb31dd8-43d9-4c62-80f0-3ef20d9717b0 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:06.798393 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbc479b-afc4-4754-ba39-ba2fbf65c633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:06.798981 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc479b-afc4-4754-ba39-ba2fbf65c633 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:06.799280 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc479b-afc4-4754-ba39-ba2fbf65c633 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:06.799550 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc479b-afc4-4754-ba39-ba2fbf65c633 None admin] PATH_INFO: `/v3/projects/d5b629fedcc945bf8c55bda4ceedc4ae` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:06.800685 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbc479b-afc4-4754-ba39-ba2fbf65c633 None admin] RBAC: Authorizing `identity:get_project(project_id=d5b629fedcc945bf8c55bda4ceedc4ae)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:06.803208 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbc479b-afc4-4754-ba39-ba2fbf65c633 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:06.805243 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 197/406] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 16:54:06 2022] GET /identity/v3/projects/d5b629fedcc945bf8c55bda4ceedc4ae => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:54:06.810649 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbc479b-afc4-4754-ba39-ba2fbf65c633 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:06.824497 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65345bf-827c-4056-9b76-0a9da721e02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:06.825234 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65345bf-827c-4056-9b76-0a9da721e02e None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:06.825453 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65345bf-827c-4056-9b76-0a9da721e02e None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:06.826241 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65345bf-827c-4056-9b76-0a9da721e02e None admin] PATH_INFO: `/v3/groups/f85bafc789824697bb18fdab47c54147` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:06.828986 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65345bf-827c-4056-9b76-0a9da721e02e None admin] RBAC: Authorizing `identity:get_group(group_id=f85bafc789824697bb18fdab47c54147)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:06.831103 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65345bf-827c-4056-9b76-0a9da721e02e None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:06.834116 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 210/407] 10.222.0.82 () {56 vars in 1180 bytes} [Thu Aug 18 16:54:06 2022] GET /identity/v3/groups/f85bafc789824697bb18fdab47c54147 => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:06.839607 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65345bf-827c-4056-9b76-0a9da721e02e None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:06.869466 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:06.963666 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbc479b-afc4-4754-ba39-ba2fbf65c633 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:06.978777 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65345bf-827c-4056-9b76-0a9da721e02e None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:06.993118 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:06.997728 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f643e71b-0f3f-4158-a5e3-7c5b135aa0c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:06.998677 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f643e71b-0f3f-4158-a5e3-7c5b135aa0c6 None admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:06.998982 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f643e71b-0f3f-4158-a5e3-7c5b135aa0c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:06.999799 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f643e71b-0f3f-4158-a5e3-7c5b135aa0c6 None admin] PATH_INFO: `/v3/projects/d5b629fedcc945bf8c55bda4ceedc4ae/groups/f85bafc789824697bb18fdab47c54147/roles/f41f6f29d9de4b1985c8c8c14f002cf2` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:07.002435 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f643e71b-0f3f-4158-a5e3-7c5b135aa0c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=d5b629fedcc945bf8c55bda4ceedc4ae, group_id=f85bafc789824697bb18fdab47c54147, role_id=f41f6f29d9de4b1985c8c8c14f002cf2)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:07.004345 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f643e71b-0f3f-4158-a5e3-7c5b135aa0c6 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:07.016707 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d05013-17a7-45d6-8dec-11426dc5a6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:07.018247 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d05013-17a7-45d6-8dec-11426dc5a6ea None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:07.018247 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d05013-17a7-45d6-8dec-11426dc5a6ea None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:07.018247 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d05013-17a7-45d6-8dec-11426dc5a6ea None admin] PATH_INFO: `/v3/projects/d5b629fedcc945bf8c55bda4ceedc4ae` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:07.019197 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d05013-17a7-45d6-8dec-11426dc5a6ea None admin] RBAC: Authorizing `identity:get_project(project_id=d5b629fedcc945bf8c55bda4ceedc4ae)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:07.020645 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d05013-17a7-45d6-8dec-11426dc5a6ea None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:07.022267 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 211/408] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 16:54:06 2022] GET /identity/v3/projects/d5b629fedcc945bf8c55bda4ceedc4ae => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:54:07.023129 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d05013-17a7-45d6-8dec-11426dc5a6ea None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:07.126427 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 198/409] 10.222.0.82 () {58 vars in 1442 bytes} [Thu Aug 18 16:54:06 2022] PUT /identity/v3/projects/d5b629fedcc945bf8c55bda4ceedc4ae/groups/f85bafc789824697bb18fdab47c54147/roles/f41f6f29d9de4b1985c8c8c14f002cf2 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 16:54:07.127165 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f643e71b-0f3f-4158-a5e3-7c5b135aa0c6 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:07.148912 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d05013-17a7-45d6-8dec-11426dc5a6ea None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:07.197291 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd7d8fb-a8bf-4b82-9a77-238bd5ab3e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:07.198874 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd7d8fb-a8bf-4b82-9a77-238bd5ab3e94 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:07.199372 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd7d8fb-a8bf-4b82-9a77-238bd5ab3e94 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:07.199842 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd7d8fb-a8bf-4b82-9a77-238bd5ab3e94 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:07.214637 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd7d8fb-a8bf-4b82-9a77-238bd5ab3e94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:54:07.221914 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-1dd7d8fb-a8bf-4b82-9a77-238bd5ab3e94 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 16:54:07.221914 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:07.221914 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:07.221914 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:07.221914 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:07.221914 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:07.221914 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:07.221914 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:07.221914 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:07.221914 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:07.221914 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:07.221914 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:07.221914 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:54:07.221914 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:54:07.221914 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:54:07.221914 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:54:07.222933 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:54:07.222933 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:54:07.222933 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:54:07.222933 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:54:07.222933 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:07.222933 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:07.222933 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:07.222933 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:07.222933 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:07.222933 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:07.222933 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:07.222933 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:07.222933 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:07.222933 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:07.222933 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:07.222933 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:07.223766 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:07.223766 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:07.223766 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:07.223766 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:07.223766 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:54:07.223766 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:54:07.223766 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:54:07.223766 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:54:07.223766 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:54:07.223766 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:54:07.223766 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 16:54:07.223766 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:07.225102 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 212/410] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 16:54:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:07.232051 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd7d8fb-a8bf-4b82-9a77-238bd5ab3e94 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:07.256257 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:07.270792 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f643e71b-0f3f-4158-a5e3-7c5b135aa0c6 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:07.288153 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:07.317901 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1746a401-8da7-4b62-9c43-d018b4ca2e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:07.318577 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1746a401-8da7-4b62-9c43-d018b4ca2e0f None admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:07.318989 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1746a401-8da7-4b62-9c43-d018b4ca2e0f None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:07.319367 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1746a401-8da7-4b62-9c43-d018b4ca2e0f None admin] PATH_INFO: `/v3/projects/d5b629fedcc945bf8c55bda4ceedc4ae/groups/f85bafc789824697bb18fdab47c54147/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:07.321410 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1746a401-8da7-4b62-9c43-d018b4ca2e0f None admin] RBAC: Authorizing `identity:create_grant(project_id=d5b629fedcc945bf8c55bda4ceedc4ae, group_id=f85bafc789824697bb18fdab47c54147, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:07.323018 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1746a401-8da7-4b62-9c43-d018b4ca2e0f None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:07.356903 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd7d8fb-a8bf-4b82-9a77-238bd5ab3e94 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:07.386723 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 199/411] 10.222.0.82 () {58 vars in 1442 bytes} [Thu Aug 18 16:54:07 2022] PUT /identity/v3/projects/d5b629fedcc945bf8c55bda4ceedc4ae/groups/f85bafc789824697bb18fdab47c54147/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 16:54:07.391766 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93031716-63bf-4ab0-90c6-dff10a26d44c None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:07.392394 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93031716-63bf-4ab0-90c6-dff10a26d44c None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:07.393088 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93031716-63bf-4ab0-90c6-dff10a26d44c None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:07.395136 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 200/412] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:54:07 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:54:07.404044 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd2a33-c20b-4c15-921d-6eb484424928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:07.407519 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd2a33-c20b-4c15-921d-6eb484424928 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:07.408087 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd2a33-c20b-4c15-921d-6eb484424928 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:07.408539 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd2a33-c20b-4c15-921d-6eb484424928 None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:07.411402 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c358119-73a4-47a0-b318-eea08ef271fa None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:07.411918 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c358119-73a4-47a0-b318-eea08ef271fa None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:07.412388 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c358119-73a4-47a0-b318-eea08ef271fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:07.417943 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cd2a33-c20b-4c15-921d-6eb484424928 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:07.420227 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cd2a33-c20b-4c15-921d-6eb484424928 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:07.423586 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cd2a33-c20b-4c15-921d-6eb484424928 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:07.448448 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 213/413] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:54:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:07.449753 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-8c358119-73a4-47a0-b318-eea08ef271fa None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:54:07.451375 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-8c358119-73a4-47a0-b318-eea08ef271fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:54:07.456062 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd2a33-c20b-4c15-921d-6eb484424928 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:07.590214 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 201/414] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:54:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 16:54:07.630834 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd2a33-c20b-4c15-921d-6eb484424928 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:07.632397 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:07.666716 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fab564-3db5-4e49-a65f-d2aaacf4a72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:07.667396 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fab564-3db5-4e49-a65f-d2aaacf4a72f None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:07.667567 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fab564-3db5-4e49-a65f-d2aaacf4a72f None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:07.668294 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fab564-3db5-4e49-a65f-d2aaacf4a72f None admin] PATH_INFO: `/v3/groups/a8bda1d009254ec6a9e5260b54e0f3d6` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:07.669505 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fab564-3db5-4e49-a65f-d2aaacf4a72f None admin] RBAC: Authorizing `identity:get_group(group_id=a8bda1d009254ec6a9e5260b54e0f3d6)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:07.671966 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fab564-3db5-4e49-a65f-d2aaacf4a72f None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:07.674297 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 214/415] 10.222.0.82 () {56 vars in 1180 bytes} [Thu Aug 18 16:54:07 2022] GET /identity/v3/groups/a8bda1d009254ec6a9e5260b54e0f3d6 => generated 229 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:07.680854 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c358119-73a4-47a0-b318-eea08ef271fa None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:07.771269 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:07.803882 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c358119-73a4-47a0-b318-eea08ef271fa None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:07.847862 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdd676a-1f99-42af-ab1c-fc9683b74766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:07.848551 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd676a-1f99-42af-ab1c-fc9683b74766 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:07.848834 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd676a-1f99-42af-ab1c-fc9683b74766 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:07.849110 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd676a-1f99-42af-ab1c-fc9683b74766 None admin] PATH_INFO: `/v3/projects/77b6b270950a499cb6b0da7c630c03b6` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:07.851628 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdd676a-1f99-42af-ab1c-fc9683b74766 None admin] RBAC: Authorizing `identity:get_project(project_id=77b6b270950a499cb6b0da7c630c03b6)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:07.853328 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdd676a-1f99-42af-ab1c-fc9683b74766 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:07.855321 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 202/416] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 16:54:07 2022] GET /identity/v3/projects/77b6b270950a499cb6b0da7c630c03b6 => generated 344 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:07.862069 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fab564-3db5-4e49-a65f-d2aaacf4a72f None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:07.996894 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fab564-3db5-4e49-a65f-d2aaacf4a72f None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:08.031534 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399bed27-335d-4fcc-b54e-227103870e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:08.032201 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399bed27-335d-4fcc-b54e-227103870e5f None admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:08.032482 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399bed27-335d-4fcc-b54e-227103870e5f None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:08.032751 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399bed27-335d-4fcc-b54e-227103870e5f None admin] PATH_INFO: `/v3/projects/77b6b270950a499cb6b0da7c630c03b6/groups/a8bda1d009254ec6a9e5260b54e0f3d6/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:08.034661 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399bed27-335d-4fcc-b54e-227103870e5f None admin] RBAC: Authorizing `identity:create_grant(project_id=77b6b270950a499cb6b0da7c630c03b6, group_id=a8bda1d009254ec6a9e5260b54e0f3d6, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:08.036120 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399bed27-335d-4fcc-b54e-227103870e5f None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:08.065850 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdd676a-1f99-42af-ab1c-fc9683b74766 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:08.132267 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 215/417] 10.222.0.82 () {58 vars in 1442 bytes} [Thu Aug 18 16:54:07 2022] PUT /identity/v3/projects/77b6b270950a499cb6b0da7c630c03b6/groups/a8bda1d009254ec6a9e5260b54e0f3d6/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 16:54:08.179498 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdd676a-1f99-42af-ab1c-fc9683b74766 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:08.216348 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf8171d-adb1-4284-82b6-fa2d48bd2ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:08.217253 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf8171d-adb1-4284-82b6-fa2d48bd2ea1 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:08.217535 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf8171d-adb1-4284-82b6-fa2d48bd2ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:08.217840 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf8171d-adb1-4284-82b6-fa2d48bd2ea1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:08.229757 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf8171d-adb1-4284-82b6-fa2d48bd2ea1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:54:08.235882 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-dbf8171d-adb1-4284-82b6-fa2d48bd2ea1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 16:54:08.235882 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:08.235882 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:08.235882 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:08.235882 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:08.235882 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:08.235882 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:08.235882 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:08.235882 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:08.235882 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:08.235882 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:08.235882 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:08.235882 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:54:08.235882 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:54:08.235882 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:54:08.235882 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:54:08.236768 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:54:08.236768 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:54:08.236768 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:54:08.236768 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:54:08.236768 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:08.236768 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:08.236768 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:08.236768 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:08.236768 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:08.236768 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:08.236768 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:08.236768 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:08.236768 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:08.236768 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:08.236768 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:08.236768 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:08.238248 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:08.238248 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:08.238248 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:08.238248 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:08.238248 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:54:08.238248 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:54:08.238248 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:54:08.238248 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:54:08.238248 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:54:08.238248 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:54:08.238248 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 16:54:08.238248 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:08.238248 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 203/418] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:54:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 16:54:08.242975 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399bed27-335d-4fcc-b54e-227103870e5f None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:08.290545 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:08.383709 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399bed27-335d-4fcc-b54e-227103870e5f None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:08.418974 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be20f809-3e6d-4e61-a17e-49f4499da998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:08.420020 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be20f809-3e6d-4e61-a17e-49f4499da998 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:08.420303 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be20f809-3e6d-4e61-a17e-49f4499da998 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:08.420573 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be20f809-3e6d-4e61-a17e-49f4499da998 None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:08.421756 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be20f809-3e6d-4e61-a17e-49f4499da998 None admin] RBAC: Adding query filter params (name=member) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:08.422057 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be20f809-3e6d-4e61-a17e-49f4499da998 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:08.423470 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be20f809-3e6d-4e61-a17e-49f4499da998 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:08.432017 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 216/419] 10.222.0.82 () {56 vars in 1101 bytes} [Thu Aug 18 16:54:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:08.438051 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf8171d-adb1-4284-82b6-fa2d48bd2ea1 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:08.451109 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff18b3-c313-4fd8-8992-a995a840cebe None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:08.451423 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff18b3-c313-4fd8-8992-a995a840cebe None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:08.451691 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff18b3-c313-4fd8-8992-a995a840cebe None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:08.455863 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 217/420] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:54:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:54:08.456365 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:08.458873 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1139b3-368b-49a6-b851-338857561115 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:08.459196 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1139b3-368b-49a6-b851-338857561115 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:08.459474 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1139b3-368b-49a6-b851-338857561115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:08.482748 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-7f1139b3-368b-49a6-b851-338857561115 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:54:08.484645 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-7f1139b3-368b-49a6-b851-338857561115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:54:08.563522 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 218/421] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:54:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:54:08.585885 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf8171d-adb1-4284-82b6-fa2d48bd2ea1 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:08.638323 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf1bfb0-5b1f-47e1-ac10-4633579e72be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:08.639237 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf1bfb0-5b1f-47e1-ac10-4633579e72be None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:08.640242 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf1bfb0-5b1f-47e1-ac10-4633579e72be None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:08.640768 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf1bfb0-5b1f-47e1-ac10-4633579e72be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:08.746557 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf1bfb0-5b1f-47e1-ac10-4633579e72be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:08.748322 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf1bfb0-5b1f-47e1-ac10-4633579e72be None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:08.767300 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:08.866035 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 204/422] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:54:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 16:54:08.871644 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1139b3-368b-49a6-b851-338857561115 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:08.900518 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:08.976301 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1139b3-368b-49a6-b851-338857561115 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:09.011624 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6465a7-dd50-45ba-ac78-0fd361fdae66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:09.012258 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6465a7-dd50-45ba-ac78-0fd361fdae66 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:09.012536 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6465a7-dd50-45ba-ac78-0fd361fdae66 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:09.012809 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6465a7-dd50-45ba-ac78-0fd361fdae66 None admin] PATH_INFO: `/v3/projects/62beed06c75a4725ae7834995533d0a0` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:09.014001 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6465a7-dd50-45ba-ac78-0fd361fdae66 None admin] RBAC: Authorizing `identity:get_project(project_id=62beed06c75a4725ae7834995533d0a0)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:09.015435 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6465a7-dd50-45ba-ac78-0fd361fdae66 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:09.017348 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 219/423] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 16:54:08 2022] GET /identity/v3/projects/62beed06c75a4725ae7834995533d0a0 => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:09.022917 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf1bfb0-5b1f-47e1-ac10-4633579e72be None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:09.103854 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6465a7-dd50-45ba-ac78-0fd361fdae66 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:09.155263 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf1bfb0-5b1f-47e1-ac10-4633579e72be None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:09.166265 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:09.211435 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a46495-28fc-4a9b-b4e3-c9a39cebd619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:09.212053 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a46495-28fc-4a9b-b4e3-c9a39cebd619 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:09.212339 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a46495-28fc-4a9b-b4e3-c9a39cebd619 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:09.214539 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a46495-28fc-4a9b-b4e3-c9a39cebd619 None admin] PATH_INFO: `/v3/groups/f85bafc789824697bb18fdab47c54147` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:09.218928 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a46495-28fc-4a9b-b4e3-c9a39cebd619 None admin] RBAC: Authorizing `identity:get_group(group_id=f85bafc789824697bb18fdab47c54147)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:09.220414 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a46495-28fc-4a9b-b4e3-c9a39cebd619 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:09.222028 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 205/424] 10.222.0.82 () {56 vars in 1180 bytes} [Thu Aug 18 16:54:09 2022] GET /identity/v3/groups/f85bafc789824697bb18fdab47c54147 => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:09.228526 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a46495-28fc-4a9b-b4e3-c9a39cebd619 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:09.229063 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:09.242822 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6465a7-dd50-45ba-ac78-0fd361fdae66 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:09.279900 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ae62ba-034d-4a05-b9a0-149f0a7546d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:09.280712 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae62ba-034d-4a05-b9a0-149f0a7546d4 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:09.280994 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae62ba-034d-4a05-b9a0-149f0a7546d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:09.281270 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae62ba-034d-4a05-b9a0-149f0a7546d4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:09.291735 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ae62ba-034d-4a05-b9a0-149f0a7546d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:54:09.296962 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-d3ae62ba-034d-4a05-b9a0-149f0a7546d4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 16:54:09.296962 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:09.296962 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:09.296962 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:09.296962 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:09.296962 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:09.296962 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:09.296962 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:09.296962 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:09.296962 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:09.296962 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:09.296962 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:09.296962 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:54:09.296962 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:54:09.296962 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:54:09.296962 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:54:09.297824 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:54:09.297824 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:54:09.297824 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:54:09.297824 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:54:09.297824 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:09.297824 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:09.297824 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:09.297824 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:09.297824 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:09.297824 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:09.297824 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:09.297824 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:09.297824 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:09.297824 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:09.297824 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:09.297824 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:09.298586 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:09.298586 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:09.298586 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:09.298586 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:09.298586 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:54:09.298586 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:54:09.298586 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:54:09.298586 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:54:09.298586 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:54:09.298586 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:54:09.298586 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 16:54:09.298586 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:09.298586 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 220/425] 10.222.0.82 () {56 vars in 1114 bytes} [Thu Aug 18 16:54:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 16:54:09.306346 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ae62ba-034d-4a05-b9a0-149f0a7546d4 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:09.384118 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a46495-28fc-4a9b-b4e3-c9a39cebd619 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:09.428369 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c6ea6f-daf1-46da-b001-cc579ab2a1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:09.428896 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6ea6f-daf1-46da-b001-cc579ab2a1d1 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:09.429174 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6ea6f-daf1-46da-b001-cc579ab2a1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:09.429358 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ae62ba-034d-4a05-b9a0-149f0a7546d4 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:09.429454 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6ea6f-daf1-46da-b001-cc579ab2a1d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:09.430148 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c6ea6f-daf1-46da-b001-cc579ab2a1d1 None admin] RBAC: Adding query filter params (group.id=f85bafc789824697bb18fdab47c54147, role.id=36d6d323d9e64e659591dbaea3081bcc, scope.project.id=62beed06c75a4725ae7834995533d0a0) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:09.430451 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c6ea6f-daf1-46da-b001-cc579ab2a1d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:09.431909 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c6ea6f-daf1-46da-b001-cc579ab2a1d1 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:09.443375 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 206/426] 10.222.0.82 () {56 vars in 1376 bytes} [Thu Aug 18 16:54:09 2022] GET /identity/v3/role_assignments?group.id=f85bafc789824697bb18fdab47c54147&scope.project.id=62beed06c75a4725ae7834995533d0a0&role.id=36d6d323d9e64e659591dbaea3081bcc => generated 611 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:09.444949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f092ee3-2986-4666-a710-6dad787b1356 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:09.445239 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f092ee3-2986-4666-a710-6dad787b1356 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:09.445512 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f092ee3-2986-4666-a710-6dad787b1356 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:09.446799 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 207/427] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:54:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:54:09.469266 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13e07b4-152f-485d-9eb0-c1479934a5d3 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:09.469580 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13e07b4-152f-485d-9eb0-c1479934a5d3 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:09.469816 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13e07b4-152f-485d-9eb0-c1479934a5d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:09.470005 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b270ee2-deef-41a2-818b-6d59df507d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:09.470629 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b270ee2-deef-41a2-818b-6d59df507d58 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:09.470924 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b270ee2-deef-41a2-818b-6d59df507d58 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:09.471281 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b270ee2-deef-41a2-818b-6d59df507d58 None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:09.471964 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b270ee2-deef-41a2-818b-6d59df507d58 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:09.472267 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b270ee2-deef-41a2-818b-6d59df507d58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:09.473686 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b270ee2-deef-41a2-818b-6d59df507d58 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:09.485209 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 221/428] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 16:54:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:54:09.491779 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b270ee2-deef-41a2-818b-6d59df507d58 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:09.520497 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-e13e07b4-152f-485d-9eb0-c1479934a5d3 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:54:09.521774 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-e13e07b4-152f-485d-9eb0-c1479934a5d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:54:09.527907 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:09.628091 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:09.641647 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b270ee2-deef-41a2-818b-6d59df507d58 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:09.666520 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 208/429] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:54:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 16:54:09.682772 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ead70fe-561f-46c7-aa11-0d5126b1b3a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:09.683334 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ead70fe-561f-46c7-aa11-0d5126b1b3a7 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:09.683612 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ead70fe-561f-46c7-aa11-0d5126b1b3a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:09.683885 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ead70fe-561f-46c7-aa11-0d5126b1b3a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:09.766577 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd89e36a-b9c4-40b1-8553-cff8b743102d None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:09.766891 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd89e36a-b9c4-40b1-8553-cff8b743102d None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:09.767368 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd89e36a-b9c4-40b1-8553-cff8b743102d None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:09.768902 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 209/430] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:54:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 16:54:09.779046 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ead70fe-561f-46c7-aa11-0d5126b1b3a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:09.779672 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858fe53-e294-4895-9f2e-980d18c1c94d None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:09.780059 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858fe53-e294-4895-9f2e-980d18c1c94d None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:09.780525 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858fe53-e294-4895-9f2e-980d18c1c94d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:09.781200 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ead70fe-561f-46c7-aa11-0d5126b1b3a7 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:09.799359 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:09.801654 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-9858fe53-e294-4895-9f2e-980d18c1c94d None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:54:09.804117 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-9858fe53-e294-4895-9f2e-980d18c1c94d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:54:09.898138 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 210/431] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:54:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 16:54:09.917433 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 222/432] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:54:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 16:54:09.923623 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858fe53-e294-4895-9f2e-980d18c1c94d None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:10.011239 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ead70fe-561f-46c7-aa11-0d5126b1b3a7 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:10.035372 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858fe53-e294-4895-9f2e-980d18c1c94d None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:10.037117 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:10.079321 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04260f1-80a3-4830-9eb9-2a2a68404c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:10.079968 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04260f1-80a3-4830-9eb9-2a2a68404c07 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:10.080252 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04260f1-80a3-4830-9eb9-2a2a68404c07 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:10.080854 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04260f1-80a3-4830-9eb9-2a2a68404c07 None admin] PATH_INFO: `/v3/projects/62beed06c75a4725ae7834995533d0a0` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:10.082470 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04260f1-80a3-4830-9eb9-2a2a68404c07 None admin] RBAC: Authorizing `identity:get_project(project_id=62beed06c75a4725ae7834995533d0a0)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:10.083982 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04260f1-80a3-4830-9eb9-2a2a68404c07 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:10.090354 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 211/433] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 16:54:09 2022] GET /identity/v3/projects/62beed06c75a4725ae7834995533d0a0 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:54:10.091845 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04260f1-80a3-4830-9eb9-2a2a68404c07 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:10.154741 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ead70fe-561f-46c7-aa11-0d5126b1b3a7 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:10.175650 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:10.204419 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570f41cf-9655-4a28-b979-d4952631d820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:10.205811 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f41cf-9655-4a28-b979-d4952631d820 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:10.206271 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f41cf-9655-4a28-b979-d4952631d820 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:10.206716 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f41cf-9655-4a28-b979-d4952631d820 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:10.216995 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04260f1-80a3-4830-9eb9-2a2a68404c07 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:10.230094 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570f41cf-9655-4a28-b979-d4952631d820 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:54:10.236936 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-570f41cf-9655-4a28-b979-d4952631d820 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 16:54:10.236936 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:10.236936 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:10.236936 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:10.236936 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:10.236936 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:10.236936 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:10.236936 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:10.236936 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:10.236936 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:10.236936 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:10.236936 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:10.236936 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:54:10.236936 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:54:10.236936 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:54:10.236936 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:54:10.237820 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:54:10.237820 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:54:10.237820 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:54:10.237820 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:54:10.237820 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:10.237820 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:10.237820 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:10.237820 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:10.237820 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:10.237820 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:10.237820 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:10.237820 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:10.237820 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:10.237820 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:10.237820 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:10.237820 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:10.239045 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:10.239045 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:10.239045 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:10.239045 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:10.239045 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:54:10.239045 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:54:10.239045 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:54:10.239045 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:54:10.239045 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:54:10.239045 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:54:10.239045 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 16:54:10.239045 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:10.239045 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 223/434] 10.222.0.82 () {56 vars in 1114 bytes} [Thu Aug 18 16:54:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:10.239045 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570f41cf-9655-4a28-b979-d4952631d820 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:10.280594 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7de82d-026a-42d3-bdb6-f3f28ea0e2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:10.281224 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7de82d-026a-42d3-bdb6-f3f28ea0e2dd None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:10.281619 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7de82d-026a-42d3-bdb6-f3f28ea0e2dd None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:10.281902 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7de82d-026a-42d3-bdb6-f3f28ea0e2dd None admin] PATH_INFO: `/v3/groups/f85bafc789824697bb18fdab47c54147` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:10.283191 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7de82d-026a-42d3-bdb6-f3f28ea0e2dd None admin] RBAC: Authorizing `identity:get_group(group_id=f85bafc789824697bb18fdab47c54147)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:10.284634 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7de82d-026a-42d3-bdb6-f3f28ea0e2dd None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:10.290412 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 212/435] 10.222.0.82 () {56 vars in 1180 bytes} [Thu Aug 18 16:54:10 2022] GET /identity/v3/groups/f85bafc789824697bb18fdab47c54147 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:10.290412 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7de82d-026a-42d3-bdb6-f3f28ea0e2dd None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:10.330889 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:10.366135 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570f41cf-9655-4a28-b979-d4952631d820 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:10.400045 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7de82d-026a-42d3-bdb6-f3f28ea0e2dd None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:10.401591 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a8be81-fd12-49a6-a296-23b2d96eac92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:10.402226 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8be81-fd12-49a6-a296-23b2d96eac92 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:10.402505 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8be81-fd12-49a6-a296-23b2d96eac92 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:10.402775 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8be81-fd12-49a6-a296-23b2d96eac92 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:10.414047 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a8be81-fd12-49a6-a296-23b2d96eac92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:54:10.418681 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-b8a8be81-fd12-49a6-a296-23b2d96eac92 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 16:54:10.418681 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:10.418681 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:10.418681 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:10.418681 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:10.418681 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:10.418681 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:10.418681 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:10.418681 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:10.418681 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:10.418681 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:10.418681 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:10.418681 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:54:10.418681 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:54:10.418681 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:54:10.418681 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:54:10.419690 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:54:10.419690 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:54:10.419690 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:54:10.419690 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:54:10.419690 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:10.419690 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:10.419690 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:10.419690 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:10.419690 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:10.419690 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:10.419690 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:10.419690 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:10.419690 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:10.419690 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:10.419690 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:10.419690 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:10.420714 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:10.420714 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:10.420714 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:10.420714 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:10.420714 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:54:10.420714 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:54:10.420714 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:54:10.420714 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:54:10.420714 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:54:10.420714 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:54:10.420714 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 16:54:10.420714 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:10.420714 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 224/436] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:54:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:10.425197 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a8be81-fd12-49a6-a296-23b2d96eac92 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:10.445461 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2008f8a6-4e4f-484c-a888-1fdb8a16d4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:10.446380 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2008f8a6-4e4f-484c-a888-1fdb8a16d4eb None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:10.447388 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2008f8a6-4e4f-484c-a888-1fdb8a16d4eb None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:10.447978 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2008f8a6-4e4f-484c-a888-1fdb8a16d4eb None admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:10.449052 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2008f8a6-4e4f-484c-a888-1fdb8a16d4eb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:10.449759 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2008f8a6-4e4f-484c-a888-1fdb8a16d4eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:10.452573 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2008f8a6-4e4f-484c-a888-1fdb8a16d4eb None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:10.457604 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:10.463526 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 213/437] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 16:54:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:10.469724 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2008f8a6-4e4f-484c-a888-1fdb8a16d4eb None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:10.528460 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a8be81-fd12-49a6-a296-23b2d96eac92 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:10.562188 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9b1bed-fcf3-40e5-898c-93048bd9b513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:10.562866 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9b1bed-fcf3-40e5-898c-93048bd9b513 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:10.563182 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9b1bed-fcf3-40e5-898c-93048bd9b513 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:10.563453 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9b1bed-fcf3-40e5-898c-93048bd9b513 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:10.564171 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9b1bed-fcf3-40e5-898c-93048bd9b513 None admin] RBAC: Adding query filter params (group.id=f85bafc789824697bb18fdab47c54147, role.id=f41f6f29d9de4b1985c8c8c14f002cf2, scope.project.id=62beed06c75a4725ae7834995533d0a0) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:10.564484 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9b1bed-fcf3-40e5-898c-93048bd9b513 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:10.564714 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:10.565925 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9b1bed-fcf3-40e5-898c-93048bd9b513 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:10.567645 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2008f8a6-4e4f-484c-a888-1fdb8a16d4eb None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:10.571591 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 225/438] 10.222.0.82 () {56 vars in 1376 bytes} [Thu Aug 18 16:54:10 2022] GET /identity/v3/role_assignments?group.id=f85bafc789824697bb18fdab47c54147&scope.project.id=62beed06c75a4725ae7834995533d0a0&role.id=f41f6f29d9de4b1985c8c8c14f002cf2 => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:10.572365 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9b1bed-fcf3-40e5-898c-93048bd9b513 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:10.618311 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b0ecb0-5c49-4f55-8cce-d67f9e9bba4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:10.618999 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b0ecb0-5c49-4f55-8cce-d67f9e9bba4a None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:10.619285 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b0ecb0-5c49-4f55-8cce-d67f9e9bba4a None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:10.619561 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b0ecb0-5c49-4f55-8cce-d67f9e9bba4a None admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:10.620256 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b0ecb0-5c49-4f55-8cce-d67f9e9bba4a None admin] RBAC: Adding query filter params (name=member) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:10.620561 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b0ecb0-5c49-4f55-8cce-d67f9e9bba4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:10.625959 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b0ecb0-5c49-4f55-8cce-d67f9e9bba4a None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:10.640576 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 214/439] 10.222.0.82 () {56 vars in 1101 bytes} [Thu Aug 18 16:54:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:10.640576 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ce039c-d57b-4c8f-957d-297005978049 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:10.640576 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ce039c-d57b-4c8f-957d-297005978049 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:10.640576 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ce039c-d57b-4c8f-957d-297005978049 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:10.640576 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 215/440] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:54:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:54:10.643919 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ce039c-d57b-4c8f-957d-297005978049 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:10.686033 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:10.693296 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9b1bed-fcf3-40e5-898c-93048bd9b513 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:10.732821 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d26cf18-0335-4c7f-9215-070e309eb3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:10.733670 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d26cf18-0335-4c7f-9215-070e309eb3f1 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:10.733971 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d26cf18-0335-4c7f-9215-070e309eb3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:10.734650 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d26cf18-0335-4c7f-9215-070e309eb3f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:10.768705 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ce039c-d57b-4c8f-957d-297005978049 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:10.812587 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564b9dd1-18f0-4129-82ea-33dce3f04cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:10.813236 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b9dd1-18f0-4129-82ea-33dce3f04cd7 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:10.813434 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b9dd1-18f0-4129-82ea-33dce3f04cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:10.813712 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b9dd1-18f0-4129-82ea-33dce3f04cd7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:10.816220 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d26cf18-0335-4c7f-9215-070e309eb3f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:10.817282 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d26cf18-0335-4c7f-9215-070e309eb3f1 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:10.852942 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:10.902671 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564b9dd1-18f0-4129-82ea-33dce3f04cd7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:10.904273 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564b9dd1-18f0-4129-82ea-33dce3f04cd7 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:10.935507 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:10.939929 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 226/441] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:54:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 16:54:10.939929 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca908f28-4499-4ce3-957c-1720e7089f98 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:10.939929 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca908f28-4499-4ce3-957c-1720e7089f98 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:10.939929 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca908f28-4499-4ce3-957c-1720e7089f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:10.959551 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-ca908f28-4499-4ce3-957c-1720e7089f98 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:54:10.961198 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-ca908f28-4499-4ce3-957c-1720e7089f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:54:11.014355 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 216/442] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:54:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 16:54:11.016223 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564b9dd1-18f0-4129-82ea-33dce3f04cd7 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:11.026893 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 227/443] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:54:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:54:11.027745 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca908f28-4499-4ce3-957c-1720e7089f98 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:11.079638 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:11.118272 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564b9dd1-18f0-4129-82ea-33dce3f04cd7 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:11.140964 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca908f28-4499-4ce3-957c-1720e7089f98 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:11.151984 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf14bfa-cf0a-4fb0-b18d-77d938b54384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:11.152609 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf14bfa-cf0a-4fb0-b18d-77d938b54384 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:11.152889 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf14bfa-cf0a-4fb0-b18d-77d938b54384 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:11.153182 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf14bfa-cf0a-4fb0-b18d-77d938b54384 None admin] PATH_INFO: `/v3/projects/d5b629fedcc945bf8c55bda4ceedc4ae` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:11.154298 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf14bfa-cf0a-4fb0-b18d-77d938b54384 None admin] RBAC: Authorizing `identity:get_project(project_id=d5b629fedcc945bf8c55bda4ceedc4ae)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:11.155937 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf14bfa-cf0a-4fb0-b18d-77d938b54384 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:11.157867 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 217/444] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 16:54:11 2022] GET /identity/v3/projects/d5b629fedcc945bf8c55bda4ceedc4ae => generated 302 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:11.162915 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf14bfa-cf0a-4fb0-b18d-77d938b54384 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:11.175063 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:11.184216 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce11d18d-619a-47d2-b673-b2e71fd47baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:11.184852 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce11d18d-619a-47d2-b673-b2e71fd47baa None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:11.185131 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce11d18d-619a-47d2-b673-b2e71fd47baa None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:11.185400 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce11d18d-619a-47d2-b673-b2e71fd47baa None admin] PATH_INFO: `/v3/projects/d5b629fedcc945bf8c55bda4ceedc4ae` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:11.187418 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce11d18d-619a-47d2-b673-b2e71fd47baa None admin] RBAC: Authorizing `identity:get_project(project_id=d5b629fedcc945bf8c55bda4ceedc4ae)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:11.188767 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce11d18d-619a-47d2-b673-b2e71fd47baa None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:11.190594 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 228/445] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 16:54:11 2022] GET /identity/v3/projects/d5b629fedcc945bf8c55bda4ceedc4ae => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:11.195664 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce11d18d-619a-47d2-b673-b2e71fd47baa None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:11.267851 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf14bfa-cf0a-4fb0-b18d-77d938b54384 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:11.299912 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:11.303512 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce11d18d-619a-47d2-b673-b2e71fd47baa None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:11.305872 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d778a37-75be-4a2b-9d32-a2fe4cdf92ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:11.306641 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d778a37-75be-4a2b-9d32-a2fe4cdf92ce None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:11.310402 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d778a37-75be-4a2b-9d32-a2fe4cdf92ce None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:11.310402 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d778a37-75be-4a2b-9d32-a2fe4cdf92ce None admin] PATH_INFO: `/v3/groups/f85bafc789824697bb18fdab47c54147` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:11.310402 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d778a37-75be-4a2b-9d32-a2fe4cdf92ce None admin] RBAC: Authorizing `identity:get_group(group_id=f85bafc789824697bb18fdab47c54147)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:11.310402 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d778a37-75be-4a2b-9d32-a2fe4cdf92ce None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:11.312087 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 218/446] 10.222.0.82 () {56 vars in 1180 bytes} [Thu Aug 18 16:54:11 2022] GET /identity/v3/groups/f85bafc789824697bb18fdab47c54147 => generated 226 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:54:11.313886 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d778a37-75be-4a2b-9d32-a2fe4cdf92ce None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:11.338780 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418ae3dc-9e89-433d-90c2-f6f76387ec4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:11.338780 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ae3dc-9e89-433d-90c2-f6f76387ec4a None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:11.338780 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ae3dc-9e89-433d-90c2-f6f76387ec4a None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:11.339163 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ae3dc-9e89-433d-90c2-f6f76387ec4a None admin] PATH_INFO: `/v3/groups/f85bafc789824697bb18fdab47c54147` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:11.340259 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418ae3dc-9e89-433d-90c2-f6f76387ec4a None admin] RBAC: Authorizing `identity:get_group(group_id=f85bafc789824697bb18fdab47c54147)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:11.341966 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418ae3dc-9e89-433d-90c2-f6f76387ec4a None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:11.343919 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 229/447] 10.222.0.82 () {56 vars in 1180 bytes} [Thu Aug 18 16:54:11 2022] GET /identity/v3/groups/f85bafc789824697bb18fdab47c54147 => generated 226 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:54:11.344667 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418ae3dc-9e89-433d-90c2-f6f76387ec4a None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:11.407580 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:11.415299 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d778a37-75be-4a2b-9d32-a2fe4cdf92ce None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:11.446193 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418ae3dc-9e89-433d-90c2-f6f76387ec4a None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:11.448480 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16110403-4e6a-49c6-bebb-e8f34cbda33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:11.449108 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16110403-4e6a-49c6-bebb-e8f34cbda33b None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:11.449390 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16110403-4e6a-49c6-bebb-e8f34cbda33b None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:11.449662 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16110403-4e6a-49c6-bebb-e8f34cbda33b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:11.459568 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16110403-4e6a-49c6-bebb-e8f34cbda33b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:54:11.464650 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-16110403-4e6a-49c6-bebb-e8f34cbda33b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 16:54:11.464650 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:11.464650 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:11.464650 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:11.464650 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:11.464650 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:11.464650 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:11.464650 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:11.464650 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:11.464650 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:11.464650 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:11.464650 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:11.464650 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:54:11.464650 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:54:11.464650 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:54:11.464650 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:54:11.465558 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:54:11.465558 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:54:11.465558 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:54:11.465558 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:54:11.465558 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:11.465558 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:11.465558 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:11.465558 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:11.465558 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:11.465558 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:11.465558 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:11.465558 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:11.465558 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:11.465558 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:11.465558 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:11.465558 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:11.466333 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:11.466333 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:11.466333 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:11.466333 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:11.466333 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:54:11.466333 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:54:11.466333 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:54:11.466333 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:54:11.466333 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:54:11.466333 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:54:11.466333 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 16:54:11.466333 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:11.467238 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 219/448] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 16:54:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:11.468657 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16110403-4e6a-49c6-bebb-e8f34cbda33b None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:11.480742 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c85aff-d682-4ba8-a2f9-32baa1c6e60a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:11.481441 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c85aff-d682-4ba8-a2f9-32baa1c6e60a None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:11.481732 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c85aff-d682-4ba8-a2f9-32baa1c6e60a None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:11.482011 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c85aff-d682-4ba8-a2f9-32baa1c6e60a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:11.482738 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c85aff-d682-4ba8-a2f9-32baa1c6e60a None admin] RBAC: Adding query filter params (group.id=f85bafc789824697bb18fdab47c54147, role.id=f41f6f29d9de4b1985c8c8c14f002cf2, scope.project.id=d5b629fedcc945bf8c55bda4ceedc4ae) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:11.483085 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c85aff-d682-4ba8-a2f9-32baa1c6e60a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:11.485067 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c85aff-d682-4ba8-a2f9-32baa1c6e60a None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:11.490723 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 230/449] 10.222.0.82 () {56 vars in 1376 bytes} [Thu Aug 18 16:54:11 2022] GET /identity/v3/role_assignments?group.id=f85bafc789824697bb18fdab47c54147&scope.project.id=d5b629fedcc945bf8c55bda4ceedc4ae&role.id=f41f6f29d9de4b1985c8c8c14f002cf2 => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:11.491548 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c85aff-d682-4ba8-a2f9-32baa1c6e60a None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:11.533239 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:11.584558 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16110403-4e6a-49c6-bebb-e8f34cbda33b None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:11.604453 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c85aff-d682-4ba8-a2f9-32baa1c6e60a None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:11.622752 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd532db-6424-4e11-b8d8-4e331e783ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:11.623407 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd532db-6424-4e11-b8d8-4e331e783ff0 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:11.623695 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd532db-6424-4e11-b8d8-4e331e783ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:11.624570 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd532db-6424-4e11-b8d8-4e331e783ff0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:11.625304 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd532db-6424-4e11-b8d8-4e331e783ff0 None admin] RBAC: Adding query filter params (group.id=f85bafc789824697bb18fdab47c54147, role.id=36d6d323d9e64e659591dbaea3081bcc, scope.project.id=d5b629fedcc945bf8c55bda4ceedc4ae) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:11.625773 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd532db-6424-4e11-b8d8-4e331e783ff0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:11.627267 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd532db-6424-4e11-b8d8-4e331e783ff0 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:11.632511 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:11.633969 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 220/450] 10.222.0.82 () {56 vars in 1376 bytes} [Thu Aug 18 16:54:11 2022] GET /identity/v3/role_assignments?group.id=f85bafc789824697bb18fdab47c54147&scope.project.id=d5b629fedcc945bf8c55bda4ceedc4ae&role.id=36d6d323d9e64e659591dbaea3081bcc => generated 611 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:11.642960 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b638bd9c-52bf-4a61-9302-5ecbad7fdef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:11.643594 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b638bd9c-52bf-4a61-9302-5ecbad7fdef0 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:11.643872 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b638bd9c-52bf-4a61-9302-5ecbad7fdef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:11.644907 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b638bd9c-52bf-4a61-9302-5ecbad7fdef0 None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:11.644907 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b638bd9c-52bf-4a61-9302-5ecbad7fdef0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:11.645085 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b638bd9c-52bf-4a61-9302-5ecbad7fdef0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:11.646499 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b638bd9c-52bf-4a61-9302-5ecbad7fdef0 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:11.657567 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 231/451] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:54:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:11.663465 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd532db-6424-4e11-b8d8-4e331e783ff0 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:11.778754 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd532db-6424-4e11-b8d8-4e331e783ff0 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:11.802844 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:11.814330 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f18c0c-b2fe-440c-88ff-73171b22da19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:11.814971 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f18c0c-b2fe-440c-88ff-73171b22da19 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:11.815259 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f18c0c-b2fe-440c-88ff-73171b22da19 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:11.825259 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f18c0c-b2fe-440c-88ff-73171b22da19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:11.884218 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f18c0c-b2fe-440c-88ff-73171b22da19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:11.885619 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f18c0c-b2fe-440c-88ff-73171b22da19 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:11.984607 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 221/452] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:54:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 16:54:11.990867 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b638bd9c-52bf-4a61-9302-5ecbad7fdef0 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:12.093484 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b638bd9c-52bf-4a61-9302-5ecbad7fdef0 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:12.127884 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1a08fe-4fb6-41f0-848b-566a754a6c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:12.128515 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1a08fe-4fb6-41f0-848b-566a754a6c41 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:12.128799 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1a08fe-4fb6-41f0-848b-566a754a6c41 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:12.129075 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1a08fe-4fb6-41f0-848b-566a754a6c41 None admin] PATH_INFO: `/v3/projects/77b6b270950a499cb6b0da7c630c03b6` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:12.130178 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1a08fe-4fb6-41f0-848b-566a754a6c41 None admin] RBAC: Authorizing `identity:get_project(project_id=77b6b270950a499cb6b0da7c630c03b6)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:12.131650 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1a08fe-4fb6-41f0-848b-566a754a6c41 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:12.133261 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 232/453] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 16:54:11 2022] GET /identity/v3/projects/77b6b270950a499cb6b0da7c630c03b6 => generated 344 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:12.139068 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f18c0c-b2fe-440c-88ff-73171b22da19 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:12.162197 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:12.237308 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f18c0c-b2fe-440c-88ff-73171b22da19 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:12.269594 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651c6199-1450-4e81-a453-7f1eb01147e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:12.270204 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c6199-1450-4e81-a453-7f1eb01147e7 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:12.270483 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c6199-1450-4e81-a453-7f1eb01147e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:12.270756 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c6199-1450-4e81-a453-7f1eb01147e7 None admin] PATH_INFO: `/v3/groups/a8bda1d009254ec6a9e5260b54e0f3d6` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:12.271873 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651c6199-1450-4e81-a453-7f1eb01147e7 None admin] RBAC: Authorizing `identity:get_group(group_id=a8bda1d009254ec6a9e5260b54e0f3d6)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:12.273315 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651c6199-1450-4e81-a453-7f1eb01147e7 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:12.274848 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 222/454] 10.222.0.82 () {56 vars in 1180 bytes} [Thu Aug 18 16:54:12 2022] GET /identity/v3/groups/a8bda1d009254ec6a9e5260b54e0f3d6 => generated 229 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:12.281292 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1a08fe-4fb6-41f0-848b-566a754a6c41 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:12.334993 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:12.377412 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1a08fe-4fb6-41f0-848b-566a754a6c41 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:12.409671 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c210c131-4a5a-437f-bc5f-784a6ced0945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:12.410187 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c210c131-4a5a-437f-bc5f-784a6ced0945 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:12.410464 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c210c131-4a5a-437f-bc5f-784a6ced0945 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:12.410737 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c210c131-4a5a-437f-bc5f-784a6ced0945 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:12.411480 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c210c131-4a5a-437f-bc5f-784a6ced0945 None admin] RBAC: Adding query filter params (group.id=a8bda1d009254ec6a9e5260b54e0f3d6, role.id=2c6966cfb10f4d72b0242e9fb5e13b17, scope.project.id=77b6b270950a499cb6b0da7c630c03b6) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:12.411787 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c210c131-4a5a-437f-bc5f-784a6ced0945 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:12.413187 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c210c131-4a5a-437f-bc5f-784a6ced0945 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:12.418609 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 233/455] 10.222.0.82 () {56 vars in 1376 bytes} [Thu Aug 18 16:54:12 2022] GET /identity/v3/role_assignments?group.id=a8bda1d009254ec6a9e5260b54e0f3d6&scope.project.id=77b6b270950a499cb6b0da7c630c03b6&role.id=2c6966cfb10f4d72b0242e9fb5e13b17 => generated 611 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:14.698776 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4935b8f-a3da-4ae8-be9c-245c8543096a None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:14.699824 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4935b8f-a3da-4ae8-be9c-245c8543096a None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:14.700302 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4935b8f-a3da-4ae8-be9c-245c8543096a None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:14.702469 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 223/456] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:54:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:54:14.710399 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa045d9-2b11-473b-a57c-493045d54712 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:14.710869 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa045d9-2b11-473b-a57c-493045d54712 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:14.710981 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa045d9-2b11-473b-a57c-493045d54712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:14.729491 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-8aa045d9-2b11-473b-a57c-493045d54712 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:54:14.730697 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-8aa045d9-2b11-473b-a57c-493045d54712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:54:14.824213 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 234/457] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:54:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:54:14.934014 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c9e7a1-1b5b-472b-9d9e-4ca6d90ed97a None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:14.934332 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c9e7a1-1b5b-472b-9d9e-4ca6d90ed97a None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:14.934573 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c9e7a1-1b5b-472b-9d9e-4ca6d90ed97a None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:14.936247 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 224/458] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:54:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:54:14.941631 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc3e39-8711-4242-a60a-f558daf5f117 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:14.941926 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc3e39-8711-4242-a60a-f558daf5f117 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:14.942210 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc3e39-8711-4242-a60a-f558daf5f117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:14.971997 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-09bc3e39-8711-4242-a60a-f558daf5f117 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:54:14.973108 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-09bc3e39-8711-4242-a60a-f558daf5f117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:54:15.041521 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:15.068526 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 235/459] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:54:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:54:15.172056 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c9e7a1-1b5b-472b-9d9e-4ca6d90ed97a None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:15.237335 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f150d5c-95f4-4881-99e3-971041c9f7ef None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:15.237625 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f150d5c-95f4-4881-99e3-971041c9f7ef None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:15.237821 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f150d5c-95f4-4881-99e3-971041c9f7ef None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:15.239350 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 236/460] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:54:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:54:15.264803 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf3662f-8f0b-488d-a306-d02a7de040d0 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:15.265083 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf3662f-8f0b-488d-a306-d02a7de040d0 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:15.265354 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf3662f-8f0b-488d-a306-d02a7de040d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:15.295010 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:15.295195 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-3bf3662f-8f0b-488d-a306-d02a7de040d0 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:54:15.295195 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-3bf3662f-8f0b-488d-a306-d02a7de040d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:54:15.318298 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c9e7a1-1b5b-472b-9d9e-4ca6d90ed97a None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:15.390127 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241448ae-8a72-4c49-bd76-23ac46e15c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:15.390770 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241448ae-8a72-4c49-bd76-23ac46e15c85 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:15.391078 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241448ae-8a72-4c49-bd76-23ac46e15c85 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:15.391352 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241448ae-8a72-4c49-bd76-23ac46e15c85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:15.403428 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 237/461] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:54:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:54:15.408188 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241448ae-8a72-4c49-bd76-23ac46e15c85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:15.409768 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241448ae-8a72-4c49-bd76-23ac46e15c85 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:15.424483 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da4fcb-8d0d-4082-bea4-f56ffee7605d None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:15.424775 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da4fcb-8d0d-4082-bea4-f56ffee7605d None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:15.425263 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da4fcb-8d0d-4082-bea4-f56ffee7605d None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:15.426919 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 238/462] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:54:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:54:15.429252 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-241448ae-8a72-4c49-bd76-23ac46e15c85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:15.429252 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:15.429252 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:15.429252 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:15.429252 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:15.429252 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:15.429252 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:15.429252 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:15.429252 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:15.429252 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:15.429252 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:15.429252 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:15.429252 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:54:15.429252 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:54:15.429252 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:54:15.429252 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:54:15.431913 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:15.431913 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:15.431913 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:15.431913 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:15.431913 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:15.431913 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:15.431913 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:15.431913 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:15.431913 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:15.431913 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:15.431913 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:15.431913 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:15.431913 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:15.431913 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:15.431913 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:15.431913 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:15.431913 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:54:15.432725 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:54:15.432725 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:15.432725 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:15.432725 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 225/463] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:54:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:15.435842 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fde035-cb41-4531-97d9-a7175aa18c38 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:15.436180 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fde035-cb41-4531-97d9-a7175aa18c38 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:15.436450 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fde035-cb41-4531-97d9-a7175aa18c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:15.440598 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241448ae-8a72-4c49-bd76-23ac46e15c85 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:15.471440 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-f3fde035-cb41-4531-97d9-a7175aa18c38 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:54:15.474049 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-f3fde035-cb41-4531-97d9-a7175aa18c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:54:15.579308 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 239/464] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:54:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 16:54:15.580549 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fde035-cb41-4531-97d9-a7175aa18c38 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:15.583018 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241448ae-8a72-4c49-bd76-23ac46e15c85 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:15.629736 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:15.671454 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebcc269-4cb5-4d70-be3b-2605fec63ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:15.672071 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebcc269-4cb5-4d70-be3b-2605fec63ad2 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:15.672364 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebcc269-4cb5-4d70-be3b-2605fec63ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:15.672626 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebcc269-4cb5-4d70-be3b-2605fec63ad2 None admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:15.673261 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebcc269-4cb5-4d70-be3b-2605fec63ad2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:15.673558 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebcc269-4cb5-4d70-be3b-2605fec63ad2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:15.675077 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebcc269-4cb5-4d70-be3b-2605fec63ad2 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:15.694866 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fde035-cb41-4531-97d9-a7175aa18c38 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:15.695525 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 226/465] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:54:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:15.701172 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebcc269-4cb5-4d70-be3b-2605fec63ad2 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:15.707160 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:15.736517 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d1ab81-c27a-4924-a0c3-13e7a77a9906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:15.737203 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d1ab81-c27a-4924-a0c3-13e7a77a9906 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:15.737462 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d1ab81-c27a-4924-a0c3-13e7a77a9906 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:15.737733 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d1ab81-c27a-4924-a0c3-13e7a77a9906 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:15.749034 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d1ab81-c27a-4924-a0c3-13e7a77a9906 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:15.750492 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d1ab81-c27a-4924-a0c3-13e7a77a9906 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:15.766476 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-09d1ab81-c27a-4924-a0c3-13e7a77a9906 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:15.766476 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:15.766476 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:15.766476 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:15.766476 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:15.766476 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:15.766476 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:15.766476 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:15.766476 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:15.766476 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:15.766476 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:15.766476 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:15.766476 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:54:15.766476 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:54:15.766476 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:54:15.766476 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:54:15.767472 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:15.767472 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:15.767472 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:15.767472 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:15.767472 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:15.767472 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:15.767472 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:15.767472 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:15.767472 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:15.767472 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:15.767472 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:15.767472 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:15.767472 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:15.767472 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:15.767472 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:15.767472 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:15.767472 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:54:15.768839 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:54:15.768839 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:15.768839 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:15.768839 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 240/466] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:54:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:15.768839 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d1ab81-c27a-4924-a0c3-13e7a77a9906 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:15.860020 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebcc269-4cb5-4d70-be3b-2605fec63ad2 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:15.869189 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:15.885945 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d1ab81-c27a-4924-a0c3-13e7a77a9906 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:15.904542 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ceabb-445a-4b95-9210-a1e6ca6b63f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:15.906699 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ceabb-445a-4b95-9210-a1e6ca6b63f5 None admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:15.907207 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ceabb-445a-4b95-9210-a1e6ca6b63f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:15.907675 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ceabb-445a-4b95-9210-a1e6ca6b63f5 None admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:15.908527 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997ceabb-445a-4b95-9210-a1e6ca6b63f5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:15.910928 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997ceabb-445a-4b95-9210-a1e6ca6b63f5 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:15.924144 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82005cb2-42ab-4be7-aede-76fd6275ec71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:15.924829 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82005cb2-42ab-4be7-aede-76fd6275ec71 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:15.925112 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82005cb2-42ab-4be7-aede-76fd6275ec71 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:15.925410 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82005cb2-42ab-4be7-aede-76fd6275ec71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:15.935483 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82005cb2-42ab-4be7-aede-76fd6275ec71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:15.937547 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82005cb2-42ab-4be7-aede-76fd6275ec71 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:15.952355 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-82005cb2-42ab-4be7-aede-76fd6275ec71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:15.952355 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:15.952355 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:15.952355 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:15.952355 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:15.952355 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:15.952355 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:15.952355 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:15.952355 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:15.952355 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:15.952355 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:15.952355 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:15.952355 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:54:15.952355 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:54:15.952355 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:54:15.952355 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:54:15.953293 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:15.953293 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:15.953293 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:15.953293 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:15.953293 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:15.953293 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:15.953293 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:15.953293 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:15.953293 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:15.953293 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:15.953293 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:15.953293 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:15.953293 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:15.953293 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:15.953293 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:15.953293 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:15.953293 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:54:15.954121 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:54:15.954121 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:15.954121 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:15.954121 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 241/467] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:54:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:15.955519 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82005cb2-42ab-4be7-aede-76fd6275ec71 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:16.000793 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 227/468] 10.222.0.82 () {60 vars in 1132 bytes} [Thu Aug 18 16:54:15 2022] POST /identity/v3/users => generated 246 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 16:54:16.002137 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ceabb-445a-4b95-9210-a1e6ca6b63f5 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:16.085833 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82005cb2-42ab-4be7-aede-76fd6275ec71 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:16.093677 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:16.139780 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a771acf0-7a17-4f5e-bd55-cc8474be22ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:16.140743 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a771acf0-7a17-4f5e-bd55-cc8474be22ae None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:16.141238 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a771acf0-7a17-4f5e-bd55-cc8474be22ae None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:16.141724 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a771acf0-7a17-4f5e-bd55-cc8474be22ae None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:16.142569 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a771acf0-7a17-4f5e-bd55-cc8474be22ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:16.143100 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a771acf0-7a17-4f5e-bd55-cc8474be22ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:16.144664 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a771acf0-7a17-4f5e-bd55-cc8474be22ae None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:16.148203 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ceabb-445a-4b95-9210-a1e6ca6b63f5 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:16.166926 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 242/469] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:54:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:16.166926 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a771acf0-7a17-4f5e-bd55-cc8474be22ae None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:16.167844 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:16.210195 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4448f42-9fb6-4312-a0f6-98ecb9031f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:16.210857 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4448f42-9fb6-4312-a0f6-98ecb9031f70 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:16.211160 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4448f42-9fb6-4312-a0f6-98ecb9031f70 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:16.211431 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4448f42-9fb6-4312-a0f6-98ecb9031f70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:16.226618 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4448f42-9fb6-4312-a0f6-98ecb9031f70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:16.228142 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4448f42-9fb6-4312-a0f6-98ecb9031f70 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:16.244046 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-c4448f42-9fb6-4312-a0f6-98ecb9031f70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:16.244046 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:16.244046 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:16.244046 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:16.244046 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:16.244046 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:16.244046 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:16.244046 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:16.244046 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:16.244046 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:16.244046 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:16.244046 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:16.244046 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:54:16.244046 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:54:16.244046 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:54:16.244046 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:54:16.244968 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:16.244968 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:16.244968 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:16.244968 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:16.244968 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:16.244968 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:16.244968 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:16.244968 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:16.244968 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:16.244968 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:16.244968 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:16.244968 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:16.244968 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:16.244968 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:16.244968 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:16.244968 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:16.244968 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:54:16.246050 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:54:16.246050 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:16.246050 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:16.246050 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 228/470] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:54:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:16.246259 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4448f42-9fb6-4312-a0f6-98ecb9031f70 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:16.294789 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a771acf0-7a17-4f5e-bd55-cc8474be22ae None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:16.331559 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ef9118-693b-4cea-ad11-b32a38620ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:16.332164 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef9118-693b-4cea-ad11-b32a38620ac2 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:16.332514 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef9118-693b-4cea-ad11-b32a38620ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:16.332791 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef9118-693b-4cea-ad11-b32a38620ac2 None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:16.333498 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ef9118-693b-4cea-ad11-b32a38620ac2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:16.333807 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ef9118-693b-4cea-ad11-b32a38620ac2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:16.335233 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ef9118-693b-4cea-ad11-b32a38620ac2 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:16.347342 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 243/471] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:54:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:16.348170 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ef9118-693b-4cea-ad11-b32a38620ac2 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:16.359607 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4448f42-9fb6-4312-a0f6-98ecb9031f70 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:16.371083 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:16.397186 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61978656-f812-4a6e-83cd-8749d17ec367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:16.397717 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61978656-f812-4a6e-83cd-8749d17ec367 None admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:16.397994 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61978656-f812-4a6e-83cd-8749d17ec367 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:16.398273 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61978656-f812-4a6e-83cd-8749d17ec367 None admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:16.398929 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61978656-f812-4a6e-83cd-8749d17ec367 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:16.400409 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61978656-f812-4a6e-83cd-8749d17ec367 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:16.414565 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:16.452226 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ef9118-693b-4cea-ad11-b32a38620ac2 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:16.486636 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069048e4-83ae-44fc-be6f-160ca94819eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:16.487263 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069048e4-83ae-44fc-be6f-160ca94819eb None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:16.487544 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069048e4-83ae-44fc-be6f-160ca94819eb None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:16.487815 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069048e4-83ae-44fc-be6f-160ca94819eb None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:16.488449 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069048e4-83ae-44fc-be6f-160ca94819eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:16.488757 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069048e4-83ae-44fc-be6f-160ca94819eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:16.490121 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069048e4-83ae-44fc-be6f-160ca94819eb None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:16.506927 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 229/472] 10.222.0.82 () {60 vars in 1132 bytes} [Thu Aug 18 16:54:16 2022] POST /identity/v3/users => generated 248 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 16:54:16.506927 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9c9bb-d6ec-46ce-8cb8-5e1c0f101350 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:16.506927 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9c9bb-d6ec-46ce-8cb8-5e1c0f101350 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:16.506927 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9c9bb-d6ec-46ce-8cb8-5e1c0f101350 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:16.506927 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 230/473] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:54:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:54:16.506927 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d9c9bb-d6ec-46ce-8cb8-5e1c0f101350 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:16.519091 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 244/474] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:54:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:16.520695 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f73ce1-3059-432c-92d6-38e09158ce2e None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:16.520988 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f73ce1-3059-432c-92d6-38e09158ce2e None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:16.521256 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f73ce1-3059-432c-92d6-38e09158ce2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:16.542962 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-49f73ce1-3059-432c-92d6-38e09158ce2e None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:54:16.544336 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-49f73ce1-3059-432c-92d6-38e09158ce2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:54:16.629807 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d9c9bb-d6ec-46ce-8cb8-5e1c0f101350 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:16.630515 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 245/475] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:54:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:54:16.631792 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f73ce1-3059-432c-92d6-38e09158ce2e None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:16.667058 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8909aa00-145c-4a24-ab9d-12906720f5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:16.667594 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8909aa00-145c-4a24-ab9d-12906720f5d3 None admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:16.667874 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8909aa00-145c-4a24-ab9d-12906720f5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:16.668143 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8909aa00-145c-4a24-ab9d-12906720f5d3 None admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:16.668775 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8909aa00-145c-4a24-ab9d-12906720f5d3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:16.670740 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8909aa00-145c-4a24-ab9d-12906720f5d3 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:16.720176 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:16.755395 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 231/476] 10.222.0.82 () {60 vars in 1132 bytes} [Thu Aug 18 16:54:16 2022] POST /identity/v3/users => generated 248 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 16:54:16.775923 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f73ce1-3059-432c-92d6-38e09158ce2e None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:16.821442 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85350bdf-a491-4db5-9c4e-cad861d88ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:16.822081 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85350bdf-a491-4db5-9c4e-cad861d88ee9 None admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:16.822365 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85350bdf-a491-4db5-9c4e-cad861d88ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:16.822640 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85350bdf-a491-4db5-9c4e-cad861d88ee9 None admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:16.823331 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85350bdf-a491-4db5-9c4e-cad861d88ee9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:16.824803 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85350bdf-a491-4db5-9c4e-cad861d88ee9 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:16.912683 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 246/477] 10.222.0.82 () {60 vars in 1132 bytes} [Thu Aug 18 16:54:16 2022] POST /identity/v3/users => generated 249 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 16:54:16.942871 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8909aa00-145c-4a24-ab9d-12906720f5d3 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:16.973390 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:17.059958 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8909aa00-145c-4a24-ab9d-12906720f5d3 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:17.095918 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c447fd0b-14f7-4f5f-b583-c17fc67354a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:17.096536 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c447fd0b-14f7-4f5f-b583-c17fc67354a7 None admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:17.096816 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c447fd0b-14f7-4f5f-b583-c17fc67354a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:17.097082 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c447fd0b-14f7-4f5f-b583-c17fc67354a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:17.097738 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c447fd0b-14f7-4f5f-b583-c17fc67354a7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:17.099208 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c447fd0b-14f7-4f5f-b583-c17fc67354a7 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:17.142871 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.sql.core [None req-c447fd0b-14f7-4f5f-b583-c17fc67354a7 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 18 16:54:17.142871 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 18 16:54:17.142871 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [parameters: {'id': 'f2f6830aeaef4cd0ac358edb8e36f85d', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 18 16:54:17.142871 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81650) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 18 16:54:17.161796 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-c447fd0b-14f7-4f5f-b583-c17fc67354a7 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 18 16:54:17.161796 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:17.161796 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 18 16:54:17.161796 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 18 16:54:17.161796 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 18 16:54:17.161796 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 18 16:54:17.161796 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 18 16:54:17.161796 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application result = self._query(query) Aug 18 16:54:17.161796 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 18 16:54:17.161796 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application conn.query(q) Aug 18 16:54:17.161796 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 18 16:54:17.161796 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 18 16:54:17.161796 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 18 16:54:17.161796 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application result.read() Aug 18 16:54:17.161796 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 18 16:54:17.162683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 18 16:54:17.162683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 18 16:54:17.162683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 18 16:54:17.162683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 18 16:54:17.162683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 18 16:54:17.162683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 18 16:54:17.162683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 18 16:54:17.162683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 18 16:54:17.162683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:17.162683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 18 16:54:17.162683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:17.162683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:17.162683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 18 16:54:17.162683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 18 16:54:17.162683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 18 16:54:17.162683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return ref.to_dict() Aug 18 16:54:17.162683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 18 16:54:17.162683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application next(self.gen) Aug 18 16:54:17.171691 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 18 16:54:17.171691 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application yield resource Aug 18 16:54:17.171691 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 18 16:54:17.171691 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application next(self.gen) Aug 18 16:54:17.171691 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 18 16:54:17.171691 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application self.session.rollback() Aug 18 16:54:17.171691 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 18 16:54:17.171691 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application self.force_reraise() Aug 18 16:54:17.171691 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 18 16:54:17.171691 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise self.value Aug 18 16:54:17.171691 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 18 16:54:17.171691 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 18 16:54:17.171691 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 18 16:54:17.171691 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application session.commit() Aug 18 16:54:17.171691 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 18 16:54:17.171691 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 18 16:54:17.171691 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 18 16:54:17.173828 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application self._prepare_impl() Aug 18 16:54:17.173828 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 18 16:54:17.173828 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application self.session.flush() Aug 18 16:54:17.173828 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 18 16:54:17.173828 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application self._flush(objects) Aug 18 16:54:17.173828 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 18 16:54:17.173828 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 18 16:54:17.173828 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 18 16:54:17.173828 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application compat.raise_( Aug 18 16:54:17.173828 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 18 16:54:17.173828 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception Aug 18 16:54:17.173828 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 18 16:54:17.173828 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application flush_context.execute() Aug 18 16:54:17.173828 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 18 16:54:17.173828 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rec.execute(self) Aug 18 16:54:17.173828 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 18 16:54:17.173828 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 18 16:54:17.175955 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 18 16:54:17.175955 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 18 16:54:17.175955 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 18 16:54:17.175955 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 18 16:54:17.175955 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 18 16:54:17.175955 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 18 16:54:17.175955 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 18 16:54:17.175955 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 18 16:54:17.175955 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 18 16:54:17.175955 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 18 16:54:17.175955 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 18 16:54:17.175955 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 18 16:54:17.175955 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 18 16:54:17.175955 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 18 16:54:17.175955 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 18 16:54:17.175955 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception Aug 18 16:54:17.178414 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 18 16:54:17.178414 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 18 16:54:17.178414 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 18 16:54:17.178414 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 18 16:54:17.178414 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 18 16:54:17.178414 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application result = self._query(query) Aug 18 16:54:17.178414 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 18 16:54:17.178414 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application conn.query(q) Aug 18 16:54:17.178414 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 18 16:54:17.178414 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 18 16:54:17.178414 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 18 16:54:17.178414 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application result.read() Aug 18 16:54:17.178414 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 18 16:54:17.178414 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 18 16:54:17.178414 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 18 16:54:17.178414 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 18 16:54:17.180370 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 18 16:54:17.180370 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 18 16:54:17.180370 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 18 16:54:17.180370 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 18 16:54:17.180370 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 18 16:54:17.180370 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 18 16:54:17.180370 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [parameters: {'id': 'f2f6830aeaef4cd0ac358edb8e36f85d', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 18 16:54:17.180370 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 18 16:54:17.180370 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:17.180370 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 16:54:17.180370 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:17.180370 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:17.180370 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:17.180370 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:17.180370 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:17.180370 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:17.184545 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:17.184545 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:17.184545 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:17.184545 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:17.184545 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:17.184545 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:17.184545 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 18 16:54:17.184545 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 18 16:54:17.184545 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:17.184545 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:17.184545 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 18 16:54:17.184545 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 18 16:54:17.184545 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 18 16:54:17.184545 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 18 16:54:17.184545 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 18 16:54:17.184545 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:17.186824 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 232/478] 10.222.0.82 () {60 vars in 1131 bytes} [Thu Aug 18 16:54:16 2022] POST /identity/v3/roles => generated 144 bytes in 221 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:17.187103 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85350bdf-a491-4db5-9c4e-cad861d88ee9 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:17.287915 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85350bdf-a491-4db5-9c4e-cad861d88ee9 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:17.315418 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:17.337451 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456da5bc-d59d-481e-af84-c33d095604a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:17.338097 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456da5bc-d59d-481e-af84-c33d095604a6 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:17.338380 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456da5bc-d59d-481e-af84-c33d095604a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:17.338658 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456da5bc-d59d-481e-af84-c33d095604a6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:17.356787 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456da5bc-d59d-481e-af84-c33d095604a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:54:17.367961 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-456da5bc-d59d-481e-af84-c33d095604a6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 16:54:17.367961 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:17.367961 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:17.367961 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:17.367961 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:17.367961 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:17.367961 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:17.367961 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:17.367961 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:17.367961 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:17.367961 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:17.367961 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:17.367961 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:54:17.367961 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:54:17.367961 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:54:17.367961 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:54:17.370204 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:54:17.370204 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:54:17.370204 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:54:17.370204 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:54:17.370204 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:17.370204 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:17.370204 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:17.370204 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:17.370204 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:17.370204 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:17.370204 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:17.370204 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:17.370204 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:17.370204 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:17.370204 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:17.370204 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:17.372155 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:17.372155 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:17.372155 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:17.372155 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:17.372155 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:54:17.372155 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:54:17.372155 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:54:17.372155 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:54:17.372155 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:54:17.372155 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:54:17.372155 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 16:54:17.372155 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:17.372155 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 247/479] 10.222.0.82 () {56 vars in 1114 bytes} [Thu Aug 18 16:54:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:17.378921 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c447fd0b-14f7-4f5f-b583-c17fc67354a7 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:17.504146 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:17.520998 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c447fd0b-14f7-4f5f-b583-c17fc67354a7 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:17.568710 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b193e55-3bdd-4816-a203-7594194bcbc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:17.569244 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b193e55-3bdd-4816-a203-7594194bcbc2 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:17.569526 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b193e55-3bdd-4816-a203-7594194bcbc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:17.569802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b193e55-3bdd-4816-a203-7594194bcbc2 None admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:17.570469 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b193e55-3bdd-4816-a203-7594194bcbc2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:17.570771 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b193e55-3bdd-4816-a203-7594194bcbc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:17.572190 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b193e55-3bdd-4816-a203-7594194bcbc2 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:17.584732 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 233/480] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 16:54:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:18.292224 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d736f0c9-418b-40a9-97a2-7f71bcb212ad None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:18.292224 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d736f0c9-418b-40a9-97a2-7f71bcb212ad None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:18.292224 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d736f0c9-418b-40a9-97a2-7f71bcb212ad None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:18.292224 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 248/481] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:54:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:54:18.311295 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8889bf-732c-4a8b-b439-ed125f01ead2 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:18.311295 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8889bf-732c-4a8b-b439-ed125f01ead2 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:18.311295 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8889bf-732c-4a8b-b439-ed125f01ead2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:18.332107 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-ed8889bf-732c-4a8b-b439-ed125f01ead2 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:54:18.333602 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-ed8889bf-732c-4a8b-b439-ed125f01ead2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:54:18.436698 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 234/482] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:54:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:54:18.850242 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d736f0c9-418b-40a9-97a2-7f71bcb212ad None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:18.988230 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b94fe-32e9-4cc2-b0dd-a3f59ce33af2 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:18.988921 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b94fe-32e9-4cc2-b0dd-a3f59ce33af2 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:18.989421 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b94fe-32e9-4cc2-b0dd-a3f59ce33af2 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:18.995751 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 235/483] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:54:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 16:54:18.997599 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa66a516-141a-462d-b9ea-6d1242eae5de None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:18.998088 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa66a516-141a-462d-b9ea-6d1242eae5de None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:18.999437 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa66a516-141a-462d-b9ea-6d1242eae5de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:19.008531 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d736f0c9-418b-40a9-97a2-7f71bcb212ad None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:19.020078 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-aa66a516-141a-462d-b9ea-6d1242eae5de None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:54:19.021560 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-aa66a516-141a-462d-b9ea-6d1242eae5de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:54:19.075664 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e404b513-080b-4142-a0fe-399499927b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:19.077559 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e404b513-080b-4142-a0fe-399499927b87 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:19.078422 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e404b513-080b-4142-a0fe-399499927b87 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:19.079540 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e404b513-080b-4142-a0fe-399499927b87 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:19.105766 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e404b513-080b-4142-a0fe-399499927b87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:54:19.116535 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-e404b513-080b-4142-a0fe-399499927b87 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 16:54:19.116535 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:19.116535 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:19.116535 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:19.116535 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:19.116535 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:19.116535 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:19.116535 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:19.116535 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:19.116535 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:19.116535 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:19.116535 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:19.116535 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:54:19.116535 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:54:19.116535 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:54:19.116535 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:54:19.117449 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:54:19.117449 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:54:19.117449 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:54:19.117449 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:54:19.117449 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:19.117449 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:19.117449 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:19.117449 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:19.117449 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:19.117449 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:19.117449 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:19.117449 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:19.117449 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:19.117449 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:19.117449 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:19.117449 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:19.118556 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:19.118556 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:19.118556 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:19.118556 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:19.118556 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:54:19.118556 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:54:19.118556 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:54:19.118556 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:54:19.118556 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:54:19.118556 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:54:19.118556 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 16:54:19.118556 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:19.125532 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 249/484] 10.222.0.82 () {56 vars in 1102 bytes} [Thu Aug 18 16:54:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 16:54:19.130249 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e404b513-080b-4142-a0fe-399499927b87 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:19.152240 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 236/485] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:54:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 16:54:19.189851 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:19.293690 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e404b513-080b-4142-a0fe-399499927b87 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:19.341775 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5afbb1-8a32-48f2-ab42-6d6e540d2e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:19.343569 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5afbb1-8a32-48f2-ab42-6d6e540d2e9f None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:19.343569 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5afbb1-8a32-48f2-ab42-6d6e540d2e9f None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:19.343569 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5afbb1-8a32-48f2-ab42-6d6e540d2e9f None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:19.343783 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5afbb1-8a32-48f2-ab42-6d6e540d2e9f None admin] RBAC: Adding query filter params (name=service) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:19.343995 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5afbb1-8a32-48f2-ab42-6d6e540d2e9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:19.345389 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5afbb1-8a32-48f2-ab42-6d6e540d2e9f None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:19.359215 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 250/486] 10.222.0.82 () {56 vars in 1103 bytes} [Thu Aug 18 16:54:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:19.364749 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa66a516-141a-462d-b9ea-6d1242eae5de None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:19.383847 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211cfb7a-2eaf-4198-89c7-15875689f8ff None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:19.384146 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211cfb7a-2eaf-4198-89c7-15875689f8ff None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:19.384416 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211cfb7a-2eaf-4198-89c7-15875689f8ff None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:19.386006 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 251/487] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:54:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:54:19.391107 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f07a83a-4590-4c46-8fca-71781c630b60 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:19.391397 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f07a83a-4590-4c46-8fca-71781c630b60 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:19.391674 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f07a83a-4590-4c46-8fca-71781c630b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:19.421406 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-3f07a83a-4590-4c46-8fca-71781c630b60 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:54:19.422651 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:19.423292 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-3f07a83a-4590-4c46-8fca-71781c630b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:54:19.499620 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 252/488] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:54:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:54:19.500979 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a37fcf-213c-4c06-9cdb-99094d9dc7ca None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:19.501261 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a37fcf-213c-4c06-9cdb-99094d9dc7ca None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:19.501589 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a37fcf-213c-4c06-9cdb-99094d9dc7ca None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:19.503612 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 253/489] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:54:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:54:19.503802 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a37fcf-213c-4c06-9cdb-99094d9dc7ca None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:19.535452 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:19.550996 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa66a516-141a-462d-b9ea-6d1242eae5de None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:19.596567 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fe7df7-dc0f-45e3-969c-4f3c22d62a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:19.598088 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe7df7-dc0f-45e3-969c-4f3c22d62a08 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:19.598624 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe7df7-dc0f-45e3-969c-4f3c22d62a08 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:19.599079 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe7df7-dc0f-45e3-969c-4f3c22d62a08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:19.613428 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fe7df7-dc0f-45e3-969c-4f3c22d62a08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:19.615993 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fe7df7-dc0f-45e3-969c-4f3c22d62a08 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:19.621751 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a37fcf-213c-4c06-9cdb-99094d9dc7ca None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:19.636038 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-c2fe7df7-dc0f-45e3-969c-4f3c22d62a08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:19.636038 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:19.636038 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:19.636038 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:19.636038 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:19.636038 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:19.636038 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:19.636038 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:19.636038 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:19.636038 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:19.636038 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:19.636038 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:19.636038 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:54:19.636038 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:54:19.636038 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:54:19.636038 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:54:19.637133 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:19.637133 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:19.637133 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:19.637133 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:19.637133 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:19.637133 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:19.637133 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:19.637133 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:19.637133 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:19.637133 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:19.637133 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:19.637133 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:19.637133 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:19.637133 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:19.637133 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:19.637133 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:19.637133 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:54:19.638148 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:54:19.638148 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:19.638148 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:19.641053 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 237/490] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:54:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:19.642467 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0968cbfe-f0d3-4c42-8d46-bf416c0acdb9 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:19.642882 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0968cbfe-f0d3-4c42-8d46-bf416c0acdb9 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:19.643306 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0968cbfe-f0d3-4c42-8d46-bf416c0acdb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:19.663272 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-0968cbfe-f0d3-4c42-8d46-bf416c0acdb9 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:54:19.664566 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-0968cbfe-f0d3-4c42-8d46-bf416c0acdb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:54:19.670725 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24c311f-1a86-4eff-abd0-f0416e460133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:19.671552 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24c311f-1a86-4eff-abd0-f0416e460133 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:19.671998 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24c311f-1a86-4eff-abd0-f0416e460133 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:19.672414 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24c311f-1a86-4eff-abd0-f0416e460133 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:19.686478 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24c311f-1a86-4eff-abd0-f0416e460133 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:54:19.693120 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-b24c311f-1a86-4eff-abd0-f0416e460133 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 16:54:19.693120 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:19.693120 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:19.693120 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:19.693120 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:19.693120 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:19.693120 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:19.693120 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:19.693120 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:19.693120 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:19.693120 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:19.693120 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:19.693120 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:54:19.693120 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:54:19.693120 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:54:19.693120 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:54:19.694143 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:54:19.694143 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:54:19.694143 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:54:19.694143 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:54:19.694143 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:19.694143 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:19.694143 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:19.694143 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:19.694143 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:19.694143 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:19.694143 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:19.694143 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:19.694143 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:19.694143 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:19.694143 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:19.694143 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:19.694943 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:19.694943 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:19.694943 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:19.694943 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:19.694943 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:54:19.694943 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:54:19.694943 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:54:19.694943 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:54:19.694943 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:54:19.694943 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:54:19.694943 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 16:54:19.694943 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:19.700762 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 254/491] 10.222.0.82 () {56 vars in 1102 bytes} [Thu Aug 18 16:54:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:19.700762 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24c311f-1a86-4eff-abd0-f0416e460133 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:19.739629 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:19.751765 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 238/492] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:54:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:54:19.752866 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0968cbfe-f0d3-4c42-8d46-bf416c0acdb9 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:19.814959 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:19.833484 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24c311f-1a86-4eff-abd0-f0416e460133 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:19.871992 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9532cd72-4e6f-4e33-b7bb-3d93f7f2ec04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:19.872884 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9532cd72-4e6f-4e33-b7bb-3d93f7f2ec04 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:19.872884 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9532cd72-4e6f-4e33-b7bb-3d93f7f2ec04 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:19.873191 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9532cd72-4e6f-4e33-b7bb-3d93f7f2ec04 None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:19.873843 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9532cd72-4e6f-4e33-b7bb-3d93f7f2ec04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:19.874151 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9532cd72-4e6f-4e33-b7bb-3d93f7f2ec04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:19.875631 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9532cd72-4e6f-4e33-b7bb-3d93f7f2ec04 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:19.887862 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 255/493] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:54:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:19.888205 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9532cd72-4e6f-4e33-b7bb-3d93f7f2ec04 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:19.908046 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0968cbfe-f0d3-4c42-8d46-bf416c0acdb9 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:19.945818 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c7bceb-71b5-465b-8e67-51fae494d33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:19.946354 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7bceb-71b5-465b-8e67-51fae494d33f None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:19.946641 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7bceb-71b5-465b-8e67-51fae494d33f None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:19.946928 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7bceb-71b5-465b-8e67-51fae494d33f None admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:19.947655 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c7bceb-71b5-465b-8e67-51fae494d33f None admin] RBAC: Adding query filter params (name=service) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:19.947956 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c7bceb-71b5-465b-8e67-51fae494d33f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:19.949748 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c7bceb-71b5-465b-8e67-51fae494d33f None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:19.963207 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 239/494] 10.222.0.82 () {56 vars in 1103 bytes} [Thu Aug 18 16:54:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:19.963207 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c7bceb-71b5-465b-8e67-51fae494d33f None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:20.006384 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9532cd72-4e6f-4e33-b7bb-3d93f7f2ec04 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:20.017389 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:20.042812 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea7b2db-7279-4990-903d-8746ef7dc041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:20.043730 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea7b2db-7279-4990-903d-8746ef7dc041 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:20.044256 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea7b2db-7279-4990-903d-8746ef7dc041 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:20.044695 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea7b2db-7279-4990-903d-8746ef7dc041 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:20.058978 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea7b2db-7279-4990-903d-8746ef7dc041 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:54:20.068075 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-6ea7b2db-7279-4990-903d-8746ef7dc041 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 16:54:20.068075 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:20.068075 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:20.068075 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:20.068075 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:20.068075 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:20.068075 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:20.068075 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:20.068075 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:20.068075 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:20.068075 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:20.068075 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:20.068075 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:54:20.068075 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:54:20.068075 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:54:20.068075 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:54:20.069119 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:54:20.069119 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:54:20.069119 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:54:20.069119 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:54:20.069119 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:20.069119 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:20.069119 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:20.069119 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:20.069119 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:20.069119 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:20.069119 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:20.069119 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:20.069119 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:20.069119 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:20.069119 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:20.069119 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:20.070140 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:20.070140 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:20.070140 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:20.070140 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:20.070140 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:54:20.070140 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:54:20.070140 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:54:20.070140 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:54:20.070140 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:54:20.070140 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:54:20.070140 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 16:54:20.070140 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:20.070864 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c7bceb-71b5-465b-8e67-51fae494d33f None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:20.073103 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 256/495] 10.222.0.82 () {56 vars in 1102 bytes} [Thu Aug 18 16:54:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:20.074106 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea7b2db-7279-4990-903d-8746ef7dc041 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:20.080228 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:20.104930 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b29ccaf-34a6-4e50-ac86-5e9221b0281e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:20.105601 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b29ccaf-34a6-4e50-ac86-5e9221b0281e None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:20.105883 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b29ccaf-34a6-4e50-ac86-5e9221b0281e None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:20.106154 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b29ccaf-34a6-4e50-ac86-5e9221b0281e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:20.114892 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b29ccaf-34a6-4e50-ac86-5e9221b0281e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:20.116378 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b29ccaf-34a6-4e50-ac86-5e9221b0281e None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:20.129759 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-6b29ccaf-34a6-4e50-ac86-5e9221b0281e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 18 16:54:20.129759 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:20.129759 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:20.129759 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:20.129759 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:20.129759 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:20.129759 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:20.129759 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:20.129759 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:20.129759 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:20.129759 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:20.129759 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:20.129759 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 16:54:20.129759 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 16:54:20.129759 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 16:54:20.129759 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 16:54:20.130673 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:20.130673 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:20.130673 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 16:54:20.130673 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:54:20.130673 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 16:54:20.130673 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:54:20.130673 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:20.130673 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:20.130673 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:20.130673 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:20.130673 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:20.130673 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:20.130673 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:20.130673 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:20.130673 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:20.130673 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:20.130673 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:20.130673 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:20.131579 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:20.131579 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:20.131579 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 16:54:20.131579 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 16:54:20.131579 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 16:54:20.131579 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 16:54:20.131579 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 16:54:20.131579 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 16:54:20.131579 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 18 16:54:20.131579 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:20.135510 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 240/496] 10.222.0.82 () {56 vars in 1093 bytes} [Thu Aug 18 16:54:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:20.136142 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b29ccaf-34a6-4e50-ac86-5e9221b0281e None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:20.189030 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea7b2db-7279-4990-903d-8746ef7dc041 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:20.224656 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3b144a-8291-4196-bbef-75f4cdf92ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:20.225361 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3b144a-8291-4196-bbef-75f4cdf92ba7 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:20.225569 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3b144a-8291-4196-bbef-75f4cdf92ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:20.225844 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3b144a-8291-4196-bbef-75f4cdf92ba7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:20.237253 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3b144a-8291-4196-bbef-75f4cdf92ba7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:20.238723 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3b144a-8291-4196-bbef-75f4cdf92ba7 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:20.242114 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b29ccaf-34a6-4e50-ac86-5e9221b0281e None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:20.254351 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-2e3b144a-8291-4196-bbef-75f4cdf92ba7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:20.254351 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:20.254351 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:20.254351 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:20.254351 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:20.254351 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:20.254351 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:20.254351 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:20.254351 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:20.254351 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:20.254351 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:20.254351 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:20.254351 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:54:20.254351 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:54:20.254351 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:54:20.254351 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:54:20.255293 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:20.255293 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:20.255293 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:20.255293 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:20.255293 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:20.255293 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:20.255293 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:20.255293 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:20.255293 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:20.255293 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:20.255293 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:20.255293 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:20.255293 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:20.255293 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:20.255293 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:20.255293 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:20.255293 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:54:20.256135 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:54:20.256135 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:20.256135 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:20.256296 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:20.256362 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 257/497] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:54:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:20.256978 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3b144a-8291-4196-bbef-75f4cdf92ba7 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:20.282887 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dde4db8-aac6-404d-b0ad-11cd69936b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:20.283567 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde4db8-aac6-404d-b0ad-11cd69936b5b None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:20.283917 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde4db8-aac6-404d-b0ad-11cd69936b5b None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:20.284197 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde4db8-aac6-404d-b0ad-11cd69936b5b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:20.294776 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dde4db8-aac6-404d-b0ad-11cd69936b5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:54:20.300903 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-5dde4db8-aac6-404d-b0ad-11cd69936b5b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 16:54:20.300903 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:20.300903 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:20.300903 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:20.300903 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:20.300903 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:20.300903 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:20.300903 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:20.300903 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:20.300903 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:20.300903 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:20.300903 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:20.300903 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:54:20.300903 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:54:20.300903 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:54:20.300903 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:54:20.301932 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:54:20.301932 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:54:20.301932 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:54:20.301932 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:54:20.301932 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:20.301932 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:20.301932 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:20.301932 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:20.301932 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:20.301932 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:20.301932 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:20.301932 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:20.301932 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:20.301932 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:20.301932 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:20.301932 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:20.302690 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:20.302690 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:20.302690 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:20.302690 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:20.302690 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:54:20.302690 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:54:20.302690 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:54:20.302690 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:54:20.302690 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:54:20.302690 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:54:20.302690 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 16:54:20.302690 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:20.302690 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 241/498] 10.222.0.82 () {56 vars in 1102 bytes} [Thu Aug 18 16:54:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:20.302690 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dde4db8-aac6-404d-b0ad-11cd69936b5b None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:20.320556 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:20.365884 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3b144a-8291-4196-bbef-75f4cdf92ba7 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:20.399044 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05f96f2-f0a6-4701-a3d3-3f6185bdffbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:20.399642 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f96f2-f0a6-4701-a3d3-3f6185bdffbd None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:20.399919 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f96f2-f0a6-4701-a3d3-3f6185bdffbd None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:20.400187 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f96f2-f0a6-4701-a3d3-3f6185bdffbd None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:20.400834 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05f96f2-f0a6-4701-a3d3-3f6185bdffbd None admin] RBAC: Adding query filter params (name=service) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:20.401130 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05f96f2-f0a6-4701-a3d3-3f6185bdffbd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:20.402195 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dde4db8-aac6-404d-b0ad-11cd69936b5b None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:20.402609 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05f96f2-f0a6-4701-a3d3-3f6185bdffbd None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:20.410298 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 258/499] 10.222.0.82 () {56 vars in 1103 bytes} [Thu Aug 18 16:54:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:20.411115 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05f96f2-f0a6-4701-a3d3-3f6185bdffbd None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:20.439186 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0467c663-e01c-4a29-994d-0b40ebc2312c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:20.439793 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0467c663-e01c-4a29-994d-0b40ebc2312c None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:20.440070 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0467c663-e01c-4a29-994d-0b40ebc2312c None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:20.440347 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0467c663-e01c-4a29-994d-0b40ebc2312c None admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:20.441055 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0467c663-e01c-4a29-994d-0b40ebc2312c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:20.441360 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0467c663-e01c-4a29-994d-0b40ebc2312c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:20.442808 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0467c663-e01c-4a29-994d-0b40ebc2312c None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:20.478582 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 242/500] 10.222.0.82 () {56 vars in 1133 bytes} [Thu Aug 18 16:54:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:20.479852 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675adb7b-64fc-4100-ba3f-b10aa4e3c91f None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:20.480148 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675adb7b-64fc-4100-ba3f-b10aa4e3c91f None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:20.480430 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675adb7b-64fc-4100-ba3f-b10aa4e3c91f None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:20.481641 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 243/501] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:54:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:54:20.482336 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675adb7b-64fc-4100-ba3f-b10aa4e3c91f None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:20.513523 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:20.516935 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05f96f2-f0a6-4701-a3d3-3f6185bdffbd None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:20.532342 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:20.548807 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fae329-2b3c-4e53-a0a1-aae80d1be2c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:20.549397 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fae329-2b3c-4e53-a0a1-aae80d1be2c8 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:20.549677 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fae329-2b3c-4e53-a0a1-aae80d1be2c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:20.549943 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fae329-2b3c-4e53-a0a1-aae80d1be2c8 None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:20.550568 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fae329-2b3c-4e53-a0a1-aae80d1be2c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:20.550866 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fae329-2b3c-4e53-a0a1-aae80d1be2c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:20.552247 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fae329-2b3c-4e53-a0a1-aae80d1be2c8 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:20.564516 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 259/502] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:54:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:20.565308 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fae329-2b3c-4e53-a0a1-aae80d1be2c8 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:20.589338 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675adb7b-64fc-4100-ba3f-b10aa4e3c91f None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:20.622426 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6924256b-9846-45ca-b088-336737453962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:20.623593 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6924256b-9846-45ca-b088-336737453962 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:20.623869 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6924256b-9846-45ca-b088-336737453962 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:20.624224 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6924256b-9846-45ca-b088-336737453962 None admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:20.624803 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6924256b-9846-45ca-b088-336737453962 None admin] RBAC: Adding query filter params (name=service) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:20.625109 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6924256b-9846-45ca-b088-336737453962 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:20.626503 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6924256b-9846-45ca-b088-336737453962 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:20.634409 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 244/503] 10.222.0.82 () {56 vars in 1103 bytes} [Thu Aug 18 16:54:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:20.635268 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6924256b-9846-45ca-b088-336737453962 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:20.663654 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fae329-2b3c-4e53-a0a1-aae80d1be2c8 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:20.696875 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7373e949-3d07-496b-83cb-e3c72d5d4919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:20.697504 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7373e949-3d07-496b-83cb-e3c72d5d4919 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:20.697789 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7373e949-3d07-496b-83cb-e3c72d5d4919 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:20.698064 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7373e949-3d07-496b-83cb-e3c72d5d4919 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:20.708584 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7373e949-3d07-496b-83cb-e3c72d5d4919 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:20.710057 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7373e949-3d07-496b-83cb-e3c72d5d4919 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:20.724558 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-7373e949-3d07-496b-83cb-e3c72d5d4919 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:20.724558 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:20.724558 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:20.724558 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:20.724558 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:20.724558 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:20.724558 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:20.724558 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:20.724558 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:20.724558 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:20.724558 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:20.724558 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:20.724558 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:54:20.724558 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:54:20.724558 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:54:20.724558 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:54:20.730288 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:20.730288 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:20.730288 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:20.730288 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:20.730288 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:20.730288 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:20.730288 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:20.730288 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:20.730288 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:20.730288 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:20.730288 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:20.730288 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:20.730288 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:20.730288 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:20.730288 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:20.730288 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:20.730288 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:54:20.731388 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:54:20.731388 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:20.731388 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:20.731388 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 260/504] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:54:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:20.731388 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be80872-55c6-4cd6-af91-0183558dc23b None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:20.731388 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be80872-55c6-4cd6-af91-0183558dc23b None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:20.731388 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be80872-55c6-4cd6-af91-0183558dc23b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:20.733038 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:20.741897 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6924256b-9846-45ca-b088-336737453962 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:20.745881 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-6be80872-55c6-4cd6-af91-0183558dc23b None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:54:20.747194 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-6be80872-55c6-4cd6-af91-0183558dc23b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:54:20.770979 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:20.776896 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dc2740-9198-4096-bb60-68ea514b2f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:20.777512 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc2740-9198-4096-bb60-68ea514b2f23 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:20.777797 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc2740-9198-4096-bb60-68ea514b2f23 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:20.778074 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc2740-9198-4096-bb60-68ea514b2f23 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:20.817845 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 261/505] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:54:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:54:20.818691 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be80872-55c6-4cd6-af91-0183558dc23b None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:20.850200 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dc2740-9198-4096-bb60-68ea514b2f23 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:20.851646 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dc2740-9198-4096-bb60-68ea514b2f23 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:20.934747 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be80872-55c6-4cd6-af91-0183558dc23b None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:20.944353 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:20.956067 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 245/506] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:54:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 16:54:20.956952 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dc2740-9198-4096-bb60-68ea514b2f23 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:20.975062 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98b8e32-357e-4dc0-a8c0-76a9e309eb40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:20.975696 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98b8e32-357e-4dc0-a8c0-76a9e309eb40 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:20.975978 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98b8e32-357e-4dc0-a8c0-76a9e309eb40 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:20.976252 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98b8e32-357e-4dc0-a8c0-76a9e309eb40 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:20.983635 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98b8e32-357e-4dc0-a8c0-76a9e309eb40 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:20.985067 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98b8e32-357e-4dc0-a8c0-76a9e309eb40 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:20.998637 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-c98b8e32-357e-4dc0-a8c0-76a9e309eb40 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 18 16:54:20.998637 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:20.998637 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:20.998637 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:20.998637 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:20.998637 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:20.998637 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:20.998637 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:20.998637 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:20.998637 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:20.998637 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:20.998637 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:20.998637 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 16:54:20.998637 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 16:54:20.998637 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 16:54:20.998637 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 16:54:20.999564 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:20.999564 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:20.999564 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 16:54:20.999564 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:54:20.999564 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 16:54:20.999564 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:54:20.999564 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:20.999564 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:20.999564 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:20.999564 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:20.999564 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:20.999564 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:20.999564 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:20.999564 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:20.999564 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:20.999564 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:20.999564 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:20.999564 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:21.000431 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:21.000431 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:21.000431 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 16:54:21.000431 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 16:54:21.000431 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 16:54:21.000431 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 16:54:21.000431 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 16:54:21.000431 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 16:54:21.000431 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 18 16:54:21.000431 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:21.001634 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 262/507] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:54:20 2022] GET /identity/v3/users/glance => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:21.002648 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98b8e32-357e-4dc0-a8c0-76a9e309eb40 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:21.022822 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:21.065618 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dc2740-9198-4096-bb60-68ea514b2f23 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:21.101784 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43733784-2761-4010-a6b2-bb47874995e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:21.102341 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43733784-2761-4010-a6b2-bb47874995e1 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:21.102623 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43733784-2761-4010-a6b2-bb47874995e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:21.102926 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43733784-2761-4010-a6b2-bb47874995e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:21.107701 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98b8e32-357e-4dc0-a8c0-76a9e309eb40 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:21.112749 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43733784-2761-4010-a6b2-bb47874995e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:21.114241 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43733784-2761-4010-a6b2-bb47874995e1 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:21.128922 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-43733784-2761-4010-a6b2-bb47874995e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:21.128922 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:21.128922 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:21.128922 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:21.128922 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:21.128922 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:21.128922 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:21.128922 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:21.128922 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:21.128922 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:21.128922 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:21.128922 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:21.128922 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:54:21.128922 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:54:21.128922 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:54:21.128922 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:54:21.129810 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:21.129810 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:21.129810 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:21.129810 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:21.129810 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:21.129810 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:21.129810 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:21.129810 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:21.129810 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:21.129810 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:21.129810 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:21.129810 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:21.129810 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:21.129810 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:21.129810 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:21.129810 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:21.129810 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:54:21.130969 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:54:21.130969 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:21.130969 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:21.130969 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 246/508] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:54:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:21.130969 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43733784-2761-4010-a6b2-bb47874995e1 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:21.142525 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd9d917-8c51-44db-b2bc-93c51cafd65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:21.143163 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd9d917-8c51-44db-b2bc-93c51cafd65d None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:21.143451 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd9d917-8c51-44db-b2bc-93c51cafd65d None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:21.143727 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd9d917-8c51-44db-b2bc-93c51cafd65d None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:21.144371 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd9d917-8c51-44db-b2bc-93c51cafd65d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:21.144676 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd9d917-8c51-44db-b2bc-93c51cafd65d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:21.146075 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd9d917-8c51-44db-b2bc-93c51cafd65d None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:21.157744 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 263/509] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:54:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:21.158539 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd9d917-8c51-44db-b2bc-93c51cafd65d None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:21.177410 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:21.232798 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43733784-2761-4010-a6b2-bb47874995e1 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:21.235236 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:21.256927 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd9d917-8c51-44db-b2bc-93c51cafd65d None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:21.268275 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db73cdfd-5ee2-4c11-a9f7-ac876572b83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:21.268909 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db73cdfd-5ee2-4c11-a9f7-ac876572b83b None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:21.269197 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db73cdfd-5ee2-4c11-a9f7-ac876572b83b None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:21.269473 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db73cdfd-5ee2-4c11-a9f7-ac876572b83b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:21.280382 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db73cdfd-5ee2-4c11-a9f7-ac876572b83b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:21.281858 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db73cdfd-5ee2-4c11-a9f7-ac876572b83b None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:21.290415 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5de1025-85b0-4795-8056-6d11b2def7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:21.291048 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5de1025-85b0-4795-8056-6d11b2def7fc None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:21.291328 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5de1025-85b0-4795-8056-6d11b2def7fc None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:21.291599 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5de1025-85b0-4795-8056-6d11b2def7fc None admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:21.292313 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5de1025-85b0-4795-8056-6d11b2def7fc None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:21.292616 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5de1025-85b0-4795-8056-6d11b2def7fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:21.294023 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5de1025-85b0-4795-8056-6d11b2def7fc None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:21.295646 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-db73cdfd-5ee2-4c11-a9f7-ac876572b83b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:21.295646 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:21.295646 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:21.295646 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:21.295646 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:21.295646 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:21.295646 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:21.295646 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:21.295646 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:21.295646 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:21.295646 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:21.295646 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:21.295646 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:54:21.295646 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:54:21.295646 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:54:21.295646 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:54:21.296691 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:21.296691 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:21.296691 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:21.296691 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:21.296691 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:21.296691 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:21.296691 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:21.296691 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:21.296691 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:21.296691 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:21.296691 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:21.296691 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:21.296691 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:21.296691 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:21.296691 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:21.296691 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:21.296691 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:54:21.297713 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:54:21.297713 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:21.297713 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:21.297713 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 247/510] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:54:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:21.297713 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db73cdfd-5ee2-4c11-a9f7-ac876572b83b None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:21.333183 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 264/511] 10.222.0.82 () {56 vars in 1137 bytes} [Thu Aug 18 16:54:21 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:21.337061 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5de1025-85b0-4795-8056-6d11b2def7fc None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:21.439232 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:21.444301 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5de1025-85b0-4795-8056-6d11b2def7fc None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:21.444708 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db73cdfd-5ee2-4c11-a9f7-ac876572b83b None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:21.477915 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4070297d-b80c-481f-8b13-55e96f47f7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:21.478505 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070297d-b80c-481f-8b13-55e96f47f7c5 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:21.478603 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f8be0f-402d-45ad-835d-0e21fe466344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:21.478780 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070297d-b80c-481f-8b13-55e96f47f7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:21.479081 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070297d-b80c-481f-8b13-55e96f47f7c5 None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:21.479209 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f8be0f-402d-45ad-835d-0e21fe466344 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:21.479493 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f8be0f-402d-45ad-835d-0e21fe466344 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:21.479710 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4070297d-b80c-481f-8b13-55e96f47f7c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:21.479797 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f8be0f-402d-45ad-835d-0e21fe466344 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:21.480008 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4070297d-b80c-481f-8b13-55e96f47f7c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:21.481376 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4070297d-b80c-481f-8b13-55e96f47f7c5 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:21.484672 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:21.491318 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f8be0f-402d-45ad-835d-0e21fe466344 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:21.492819 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f8be0f-402d-45ad-835d-0e21fe466344 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:21.493830 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 265/512] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:54:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:21.494840 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4070297d-b80c-481f-8b13-55e96f47f7c5 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:21.507079 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-d7f8be0f-402d-45ad-835d-0e21fe466344 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:21.507079 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:21.507079 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:21.507079 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:21.507079 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:21.507079 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:21.507079 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:21.507079 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:21.507079 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:21.507079 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:21.507079 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:21.507079 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:21.507079 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:54:21.507079 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:54:21.507079 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:54:21.507079 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:54:21.508549 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:21.508549 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:21.508549 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:21.508549 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:21.508549 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:21.508549 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:21.508549 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:21.508549 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:21.508549 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:21.508549 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:21.508549 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:21.508549 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:21.508549 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:21.508549 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:21.508549 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:21.508549 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:21.508549 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:54:21.509359 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:54:21.509359 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:21.509359 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:21.509359 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 248/513] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:54:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:21.509359 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f8be0f-402d-45ad-835d-0e21fe466344 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:21.594840 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4070297d-b80c-481f-8b13-55e96f47f7c5 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:21.609230 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f8be0f-402d-45ad-835d-0e21fe466344 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:21.627311 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f58d78-447c-4000-9054-0ebcb4178252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:21.627932 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f58d78-447c-4000-9054-0ebcb4178252 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:21.628213 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f58d78-447c-4000-9054-0ebcb4178252 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:21.628480 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f58d78-447c-4000-9054-0ebcb4178252 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:21.635656 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f58d78-447c-4000-9054-0ebcb4178252 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:21.637086 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f58d78-447c-4000-9054-0ebcb4178252 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:21.641840 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd5ac9b-671a-4247-adfb-31f7778d8906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:21.642441 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5ac9b-671a-4247-adfb-31f7778d8906 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:21.642724 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5ac9b-671a-4247-adfb-31f7778d8906 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:21.643033 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5ac9b-671a-4247-adfb-31f7778d8906 None admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:21.643670 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd5ac9b-671a-4247-adfb-31f7778d8906 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:21.643974 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd5ac9b-671a-4247-adfb-31f7778d8906 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:21.645345 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd5ac9b-671a-4247-adfb-31f7778d8906 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:21.648681 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-27f58d78-447c-4000-9054-0ebcb4178252 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 18 16:54:21.648681 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:21.648681 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:21.648681 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:21.648681 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:21.648681 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:21.648681 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:21.648681 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:21.648681 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:21.648681 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:21.648681 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:21.648681 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:21.648681 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 16:54:21.648681 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 16:54:21.648681 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 16:54:21.648681 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 16:54:21.649666 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:21.649666 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:21.649666 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 16:54:21.649666 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:54:21.649666 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 16:54:21.649666 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:54:21.649666 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:21.649666 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:21.649666 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:21.649666 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:21.649666 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:21.649666 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:21.649666 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:21.649666 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:21.649666 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:21.649666 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:21.649666 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:21.649666 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:21.655824 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:21.655824 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:21.655824 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 16:54:21.655824 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 16:54:21.655824 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 16:54:21.655824 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 16:54:21.655824 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 16:54:21.655824 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 16:54:21.655824 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 18 16:54:21.655824 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:21.655824 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 266/514] 10.222.0.82 () {56 vars in 1102 bytes} [Thu Aug 18 16:54:21 2022] GET /identity/v3/users/neutron => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:21.655824 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f58d78-447c-4000-9054-0ebcb4178252 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:21.662291 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 249/515] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:54:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:21.663451 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd5ac9b-671a-4247-adfb-31f7778d8906 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:21.674293 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:21.707506 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:21.751967 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f58d78-447c-4000-9054-0ebcb4178252 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:21.763111 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd5ac9b-671a-4247-adfb-31f7778d8906 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:21.786442 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09ae8fd-e02f-4bf4-969e-0b875ecc97e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:21.787082 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09ae8fd-e02f-4bf4-969e-0b875ecc97e7 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:21.787363 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09ae8fd-e02f-4bf4-969e-0b875ecc97e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:21.787631 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09ae8fd-e02f-4bf4-969e-0b875ecc97e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:21.800096 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209016d0-a31d-46ce-998a-930edb6d7623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:21.800726 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209016d0-a31d-46ce-998a-930edb6d7623 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:21.801007 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209016d0-a31d-46ce-998a-930edb6d7623 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:21.801281 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209016d0-a31d-46ce-998a-930edb6d7623 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:21.808382 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209016d0-a31d-46ce-998a-930edb6d7623 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:21.809837 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209016d0-a31d-46ce-998a-930edb6d7623 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:21.821799 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-209016d0-a31d-46ce-998a-930edb6d7623 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 18 16:54:21.821799 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:21.821799 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:21.821799 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:21.821799 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:21.821799 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:21.821799 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:21.821799 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:21.821799 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:21.821799 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:21.821799 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:21.821799 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:21.821799 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 16:54:21.821799 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 16:54:21.821799 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 16:54:21.821799 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 16:54:21.822687 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:21.822687 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:21.822687 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 16:54:21.822687 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:54:21.822687 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 16:54:21.822687 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:54:21.822687 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:21.822687 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:21.822687 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:21.822687 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:21.822687 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:21.822687 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:21.822687 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:21.822687 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:21.822687 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:21.822687 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:21.822687 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:21.822687 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:21.828470 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:21.828470 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:21.828470 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 16:54:21.828470 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 16:54:21.828470 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 16:54:21.828470 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 16:54:21.828470 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 16:54:21.828470 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 16:54:21.828470 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 18 16:54:21.828470 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:21.828470 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 250/516] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:54:21 2022] GET /identity/v3/users/cinder => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:21.828470 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209016d0-a31d-46ce-998a-930edb6d7623 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:21.855796 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09ae8fd-e02f-4bf4-969e-0b875ecc97e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:21.857252 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09ae8fd-e02f-4bf4-969e-0b875ecc97e7 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:21.887715 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:21.910048 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:21.921241 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209016d0-a31d-46ce-998a-930edb6d7623 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:21.953885 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 267/517] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:54:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 16:54:21.954823 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09ae8fd-e02f-4bf4-969e-0b875ecc97e7 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:21.954980 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad06e621-77f7-4f2e-b61a-f37d3de1fc53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:21.955469 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad06e621-77f7-4f2e-b61a-f37d3de1fc53 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:21.955747 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad06e621-77f7-4f2e-b61a-f37d3de1fc53 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:21.956020 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad06e621-77f7-4f2e-b61a-f37d3de1fc53 None admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:21.956647 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad06e621-77f7-4f2e-b61a-f37d3de1fc53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:21.956996 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad06e621-77f7-4f2e-b61a-f37d3de1fc53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:21.958386 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad06e621-77f7-4f2e-b61a-f37d3de1fc53 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:21.970643 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 251/518] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:54:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:21.971475 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad06e621-77f7-4f2e-b61a-f37d3de1fc53 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:22.053945 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09ae8fd-e02f-4bf4-969e-0b875ecc97e7 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:22.068899 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad06e621-77f7-4f2e-b61a-f37d3de1fc53 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:22.087042 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b3f64-a9f1-4d42-b045-1ed5d8062965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:22.087643 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b3f64-a9f1-4d42-b045-1ed5d8062965 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:22.087928 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b3f64-a9f1-4d42-b045-1ed5d8062965 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:22.088199 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b3f64-a9f1-4d42-b045-1ed5d8062965 None admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:22.088911 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6b3f64-a9f1-4d42-b045-1ed5d8062965 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:22.089213 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6b3f64-a9f1-4d42-b045-1ed5d8062965 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:22.090624 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6b3f64-a9f1-4d42-b045-1ed5d8062965 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:22.094801 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:22.102095 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e84b6bd-1fcb-428f-812a-e7998d394081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:22.105489 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e84b6bd-1fcb-428f-812a-e7998d394081 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:22.105489 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e84b6bd-1fcb-428f-812a-e7998d394081 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:22.105489 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e84b6bd-1fcb-428f-812a-e7998d394081 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:22.110073 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 268/519] 10.222.0.82 () {56 vars in 1139 bytes} [Thu Aug 18 16:54:21 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:22.110332 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e84b6bd-1fcb-428f-812a-e7998d394081 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:22.110650 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b3f64-a9f1-4d42-b045-1ed5d8062965 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:22.111737 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e84b6bd-1fcb-428f-812a-e7998d394081 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:22.124357 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-6e84b6bd-1fcb-428f-812a-e7998d394081 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:54:22.124357 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:22.124357 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:22.124357 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:22.124357 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:22.124357 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:22.124357 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:22.124357 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:22.124357 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:22.124357 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:22.124357 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:22.124357 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:22.124357 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 16:54:22.124357 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 16:54:22.124357 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 16:54:22.125315 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 16:54:22.125315 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:22.125315 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:22.125315 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:22.125315 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:22.125315 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:22.125315 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:22.125315 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:22.125315 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:22.125315 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:22.125315 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:22.125315 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:22.125315 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:22.125315 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:22.125315 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:22.125315 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:22.125315 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:22.126197 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 16:54:22.126197 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 16:54:22.126197 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 16:54:22.126197 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 16:54:22.126197 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 16:54:22.126197 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 16:54:22.126197 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:54:22.126197 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:22.126197 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 252/520] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 16:54:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:22.126831 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e84b6bd-1fcb-428f-812a-e7998d394081 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:22.127255 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:22.211729 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b3f64-a9f1-4d42-b045-1ed5d8062965 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:22.225642 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e84b6bd-1fcb-428f-812a-e7998d394081 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:22.245849 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d45ff1-233d-49bd-8387-8da3aef5fb59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:22.246447 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d45ff1-233d-49bd-8387-8da3aef5fb59 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:22.246730 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d45ff1-233d-49bd-8387-8da3aef5fb59 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:22.247137 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d45ff1-233d-49bd-8387-8da3aef5fb59 None admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:22.247888 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d45ff1-233d-49bd-8387-8da3aef5fb59 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:22.248189 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d45ff1-233d-49bd-8387-8da3aef5fb59 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:22.249653 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d45ff1-233d-49bd-8387-8da3aef5fb59 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:22.258461 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143ce642-92c7-421b-8e25-d4e80bbf6deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:22.259112 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143ce642-92c7-421b-8e25-d4e80bbf6deb None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:22.259391 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143ce642-92c7-421b-8e25-d4e80bbf6deb None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:22.259660 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143ce642-92c7-421b-8e25-d4e80bbf6deb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:22.266076 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 269/521] 10.222.0.82 () {56 vars in 1137 bytes} [Thu Aug 18 16:54:22 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:22.266853 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d45ff1-233d-49bd-8387-8da3aef5fb59 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:22.270429 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143ce642-92c7-421b-8e25-d4e80bbf6deb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:22.272193 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143ce642-92c7-421b-8e25-d4e80bbf6deb None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:22.286645 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-143ce642-92c7-421b-8e25-d4e80bbf6deb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:22.286645 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:22.286645 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:22.286645 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:22.286645 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:22.286645 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:22.286645 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:22.286645 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:22.286645 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:22.286645 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:22.286645 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:22.286645 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:22.286645 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:54:22.286645 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:54:22.286645 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:54:22.286645 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:54:22.287728 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:22.287728 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:22.287728 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:22.287728 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:22.287728 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:22.287728 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:22.287728 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:22.287728 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:22.287728 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:22.287728 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:22.287728 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:22.287728 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:22.287728 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:22.287728 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:22.287728 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:22.287728 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:22.287728 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:54:22.288641 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:54:22.288641 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:22.288641 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:22.288641 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 253/522] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:54:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:22.288988 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143ce642-92c7-421b-8e25-d4e80bbf6deb None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:22.329219 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:22.341189 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:22.371238 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d45ff1-233d-49bd-8387-8da3aef5fb59 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:22.384724 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143ce642-92c7-421b-8e25-d4e80bbf6deb None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:22.405514 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60c5798-1cfb-47a1-8c12-34e053982a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:22.406105 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c5798-1cfb-47a1-8c12-34e053982a7f None admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:22.406382 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c5798-1cfb-47a1-8c12-34e053982a7f None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:22.406652 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c5798-1cfb-47a1-8c12-34e053982a7f None admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:22.407313 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60c5798-1cfb-47a1-8c12-34e053982a7f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:22.408724 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60c5798-1cfb-47a1-8c12-34e053982a7f None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:22.417953 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a4d5b7-5439-4be9-833b-e640689dbde2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:22.418561 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a4d5b7-5439-4be9-833b-e640689dbde2 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:22.418841 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a4d5b7-5439-4be9-833b-e640689dbde2 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:22.419151 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a4d5b7-5439-4be9-833b-e640689dbde2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:22.485683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a4d5b7-5439-4be9-833b-e640689dbde2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:22.487138 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a4d5b7-5439-4be9-833b-e640689dbde2 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:22.487838 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 270/523] 10.222.0.82 () {60 vars in 1132 bytes} [Thu Aug 18 16:54:22 2022] POST /identity/v3/users => generated 247 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 16:54:22.488668 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60c5798-1cfb-47a1-8c12-34e053982a7f None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:22.534185 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:22.600655 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 254/524] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:54:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 16:54:22.601326 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a4d5b7-5439-4be9-833b-e640689dbde2 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:22.627224 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60c5798-1cfb-47a1-8c12-34e053982a7f None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:22.657874 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:22.666283 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db446dab-4487-4ed9-8710-9b7e10f7be51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:22.666890 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db446dab-4487-4ed9-8710-9b7e10f7be51 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:22.667197 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db446dab-4487-4ed9-8710-9b7e10f7be51 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:22.667465 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db446dab-4487-4ed9-8710-9b7e10f7be51 None admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:22.668151 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db446dab-4487-4ed9-8710-9b7e10f7be51 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:22.670250 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db446dab-4487-4ed9-8710-9b7e10f7be51 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:22.670250 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db446dab-4487-4ed9-8710-9b7e10f7be51 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:22.691617 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 271/525] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 16:54:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:22.692951 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db446dab-4487-4ed9-8710-9b7e10f7be51 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:22.717515 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a4d5b7-5439-4be9-833b-e640689dbde2 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:22.752315 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3695c2-064c-471e-9e4a-4112dcbf13d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:22.752851 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3695c2-064c-471e-9e4a-4112dcbf13d5 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:22.753140 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3695c2-064c-471e-9e4a-4112dcbf13d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:22.753412 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3695c2-064c-471e-9e4a-4112dcbf13d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:22.768147 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:22.817462 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db446dab-4487-4ed9-8710-9b7e10f7be51 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:22.831107 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3695c2-064c-471e-9e4a-4112dcbf13d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:22.832898 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3695c2-064c-471e-9e4a-4112dcbf13d5 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:22.852374 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ce4aa7-0e87-45f1-8a27-f9391d16bb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:22.852981 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ce4aa7-0e87-45f1-8a27-f9391d16bb29 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:22.853285 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ce4aa7-0e87-45f1-8a27-f9391d16bb29 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:22.853557 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ce4aa7-0e87-45f1-8a27-f9391d16bb29 None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:22.854189 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ce4aa7-0e87-45f1-8a27-f9391d16bb29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:22.854486 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ce4aa7-0e87-45f1-8a27-f9391d16bb29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:22.855882 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ce4aa7-0e87-45f1-8a27-f9391d16bb29 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:22.872427 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 272/526] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:54:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:22.873891 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ce4aa7-0e87-45f1-8a27-f9391d16bb29 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:22.926692 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:22.941050 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 255/527] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:54:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 16:54:22.946445 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3695c2-064c-471e-9e4a-4112dcbf13d5 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:22.982177 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ce4aa7-0e87-45f1-8a27-f9391d16bb29 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:22.994435 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:23.019357 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b5d80e-ab8c-43aa-a762-67b0991559a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:23.019979 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5d80e-ab8c-43aa-a762-67b0991559a8 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:23.020466 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5d80e-ab8c-43aa-a762-67b0991559a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:23.020812 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5d80e-ab8c-43aa-a762-67b0991559a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:23.032277 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b5d80e-ab8c-43aa-a762-67b0991559a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:23.033867 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b5d80e-ab8c-43aa-a762-67b0991559a8 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:23.049442 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-63b5d80e-ab8c-43aa-a762-67b0991559a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:23.049442 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:23.049442 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:23.049442 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:23.049442 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:23.049442 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:23.049442 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:23.049442 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:23.049442 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:23.049442 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:23.049442 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:23.049442 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:23.049442 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:54:23.049442 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:54:23.049442 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:54:23.049442 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:54:23.050510 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:23.050510 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:23.050510 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:23.050510 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:23.050510 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:23.050510 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:23.050510 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:23.050510 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:23.050510 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:23.050510 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:23.050510 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:23.050510 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:23.050510 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:23.050510 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:23.050510 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:23.050510 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:23.050510 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:54:23.051402 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:54:23.051402 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:23.051402 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:23.051402 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 273/528] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:54:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:23.051402 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b5d80e-ab8c-43aa-a762-67b0991559a8 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:23.066254 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3695c2-064c-471e-9e4a-4112dcbf13d5 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:23.102328 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9a7d31-b5a4-4c67-8474-eb296768abf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:23.103070 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9a7d31-b5a4-4c67-8474-eb296768abf6 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:23.103412 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9a7d31-b5a4-4c67-8474-eb296768abf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:23.103752 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9a7d31-b5a4-4c67-8474-eb296768abf6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:23.104532 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9a7d31-b5a4-4c67-8474-eb296768abf6 None admin] RBAC: Adding query filter params (role.id=5e0f402e1af84b44a723ee0730315d74, scope.project.id=2293c0aab61f44849d0d48b214cde7ea, user.id=55723c756cae4e7ab636054db48de02d) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:23.104902 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9a7d31-b5a4-4c67-8474-eb296768abf6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:23.106414 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9a7d31-b5a4-4c67-8474-eb296768abf6 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:23.123040 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 256/529] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 16:54:22 2022] GET /identity/v3/role_assignments?user.id=55723c756cae4e7ab636054db48de02d&scope.project.id=2293c0aab61f44849d0d48b214cde7ea&role.id=5e0f402e1af84b44a723ee0730315d74 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:23.123040 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9a7d31-b5a4-4c67-8474-eb296768abf6 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:23.176565 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:23.180436 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b5d80e-ab8c-43aa-a762-67b0991559a8 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:23.228670 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d44eb2-e561-4f60-aac3-397c2705040d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:23.229530 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d44eb2-e561-4f60-aac3-397c2705040d None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:23.229809 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d44eb2-e561-4f60-aac3-397c2705040d None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:23.230076 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d44eb2-e561-4f60-aac3-397c2705040d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:23.240261 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9a7d31-b5a4-4c67-8474-eb296768abf6 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:23.243718 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:23.247691 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d44eb2-e561-4f60-aac3-397c2705040d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:23.250783 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d44eb2-e561-4f60-aac3-397c2705040d None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:23.268929 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-a2d44eb2-e561-4f60-aac3-397c2705040d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:54:23.268929 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:23.268929 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:23.268929 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:23.268929 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:23.268929 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:23.268929 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:23.268929 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:23.268929 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:23.268929 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:23.268929 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:23.268929 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:23.268929 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 16:54:23.268929 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 16:54:23.268929 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 16:54:23.270316 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 16:54:23.270316 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:23.270316 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:23.270316 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:23.270316 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:23.270316 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:23.270316 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:23.270316 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:23.270316 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:23.270316 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:23.270316 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:23.270316 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:23.270316 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:23.270316 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:23.270316 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:23.270316 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:23.270316 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:23.271365 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 16:54:23.271365 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 16:54:23.271365 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 16:54:23.271365 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 16:54:23.271365 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 16:54:23.271365 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 16:54:23.271365 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:54:23.271365 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:23.273746 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 274/530] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 16:54:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:23.275539 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d44eb2-e561-4f60-aac3-397c2705040d None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:23.285803 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555fd61b-1f5b-4866-a2f4-3048837978d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:23.288928 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555fd61b-1f5b-4866-a2f4-3048837978d0 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:23.289321 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555fd61b-1f5b-4866-a2f4-3048837978d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:23.289694 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555fd61b-1f5b-4866-a2f4-3048837978d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:23.301762 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555fd61b-1f5b-4866-a2f4-3048837978d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:23.303404 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555fd61b-1f5b-4866-a2f4-3048837978d0 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:23.318936 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-555fd61b-1f5b-4866-a2f4-3048837978d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:23.318936 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:23.318936 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:23.318936 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:23.318936 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:23.318936 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:23.318936 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:23.318936 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:23.318936 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:23.318936 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:23.318936 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:23.318936 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:23.318936 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:54:23.318936 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:54:23.318936 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:54:23.318936 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:54:23.320032 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:23.320032 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:23.320032 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:23.320032 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:23.320032 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:23.320032 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:23.320032 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:23.320032 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:23.320032 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:23.320032 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:23.320032 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:23.320032 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:23.320032 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:23.320032 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:23.320032 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:23.320032 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:23.320032 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:54:23.321026 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:54:23.321026 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:23.321026 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:23.322391 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 257/531] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:54:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:23.323362 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555fd61b-1f5b-4866-a2f4-3048837978d0 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:23.403653 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d44eb2-e561-4f60-aac3-397c2705040d None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:23.441358 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e9c756-ca41-4533-a4ad-a56ca239aa34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:23.442120 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e9c756-ca41-4533-a4ad-a56ca239aa34 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:23.442492 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e9c756-ca41-4533-a4ad-a56ca239aa34 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:23.442925 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e9c756-ca41-4533-a4ad-a56ca239aa34 None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:23.443712 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e9c756-ca41-4533-a4ad-a56ca239aa34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:23.444106 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e9c756-ca41-4533-a4ad-a56ca239aa34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:23.445639 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e9c756-ca41-4533-a4ad-a56ca239aa34 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:23.461991 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 275/532] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:54:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:23.461991 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e9c756-ca41-4533-a4ad-a56ca239aa34 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:23.462807 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555fd61b-1f5b-4866-a2f4-3048837978d0 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:23.494545 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:23.504235 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fc2b73-5cbf-4a75-b450-579ebf864d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:23.504877 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc2b73-5cbf-4a75-b450-579ebf864d2b None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:23.505139 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc2b73-5cbf-4a75-b450-579ebf864d2b None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:23.505418 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc2b73-5cbf-4a75-b450-579ebf864d2b None admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:23.506488 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fc2b73-5cbf-4a75-b450-579ebf864d2b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:23.506803 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fc2b73-5cbf-4a75-b450-579ebf864d2b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:23.508572 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fc2b73-5cbf-4a75-b450-579ebf864d2b None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:23.532791 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 258/533] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 16:54:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:23.533853 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fc2b73-5cbf-4a75-b450-579ebf864d2b None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:23.559893 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:23.581359 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e9c756-ca41-4533-a4ad-a56ca239aa34 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:23.621427 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca5493-b5a3-433f-8031-308605ecacce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:23.622451 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca5493-b5a3-433f-8031-308605ecacce None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:23.623501 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca5493-b5a3-433f-8031-308605ecacce None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:23.623997 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca5493-b5a3-433f-8031-308605ecacce None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:23.624952 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffca5493-b5a3-433f-8031-308605ecacce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:23.625478 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffca5493-b5a3-433f-8031-308605ecacce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:23.627380 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffca5493-b5a3-433f-8031-308605ecacce None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:23.648445 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 276/534] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:54:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:23.649372 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca5493-b5a3-433f-8031-308605ecacce None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:23.663097 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fc2b73-5cbf-4a75-b450-579ebf864d2b None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:23.703446 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4236f7-0650-4b70-ae18-c4b9650a8d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:23.704098 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4236f7-0650-4b70-ae18-c4b9650a8d71 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:23.704381 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4236f7-0650-4b70-ae18-c4b9650a8d71 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:23.704707 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4236f7-0650-4b70-ae18-c4b9650a8d71 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:23.713784 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4236f7-0650-4b70-ae18-c4b9650a8d71 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:23.715691 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4236f7-0650-4b70-ae18-c4b9650a8d71 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:23.730517 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-0a4236f7-0650-4b70-ae18-c4b9650a8d71 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:54:23.730517 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:23.730517 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:23.730517 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:23.730517 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:23.730517 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:23.730517 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:23.730517 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:23.730517 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:23.730517 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:23.730517 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:23.730517 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:23.730517 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 16:54:23.730517 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 16:54:23.730517 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 16:54:23.731356 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 16:54:23.731356 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:23.731356 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:23.731356 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:23.731356 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:23.731356 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:23.731356 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:23.731356 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:23.731356 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:23.731356 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:23.731356 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:23.731356 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:23.731356 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:23.731356 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:23.731356 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:23.731356 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:23.731356 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:23.732232 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 16:54:23.732232 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 16:54:23.732232 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 16:54:23.732232 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 16:54:23.732232 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 16:54:23.732232 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 16:54:23.732232 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:54:23.732232 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:23.732232 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 259/535] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 16:54:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:23.733270 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4236f7-0650-4b70-ae18-c4b9650a8d71 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:23.750838 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:23.777795 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca5493-b5a3-433f-8031-308605ecacce None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:23.828621 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:23.828952 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793d6d33-e53a-46ad-8533-140dbce9779b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:23.829829 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793d6d33-e53a-46ad-8533-140dbce9779b None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:23.830186 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793d6d33-e53a-46ad-8533-140dbce9779b None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:23.830456 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793d6d33-e53a-46ad-8533-140dbce9779b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:23.831209 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793d6d33-e53a-46ad-8533-140dbce9779b None admin] RBAC: Adding query filter params (role.id=5e0f402e1af84b44a723ee0730315d74, scope.project.id=2293c0aab61f44849d0d48b214cde7ea, user.id=2bee1ba6bb324de4915904f83349b3c4) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:23.831517 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793d6d33-e53a-46ad-8533-140dbce9779b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:23.833566 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793d6d33-e53a-46ad-8533-140dbce9779b None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:23.841022 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 277/536] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 16:54:23 2022] GET /identity/v3/role_assignments?user.id=2bee1ba6bb324de4915904f83349b3c4&scope.project.id=2293c0aab61f44849d0d48b214cde7ea&role.id=5e0f402e1af84b44a723ee0730315d74 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:23.842718 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793d6d33-e53a-46ad-8533-140dbce9779b None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:23.859806 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4236f7-0650-4b70-ae18-c4b9650a8d71 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:23.910299 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2a0c31-2814-48d3-a530-031d27778861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:23.911239 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2a0c31-2814-48d3-a530-031d27778861 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:23.911527 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2a0c31-2814-48d3-a530-031d27778861 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:23.911805 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2a0c31-2814-48d3-a530-031d27778861 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:23.922161 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2a0c31-2814-48d3-a530-031d27778861 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:23.924022 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2a0c31-2814-48d3-a530-031d27778861 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:23.937632 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-1e2a0c31-2814-48d3-a530-031d27778861 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:54:23.937632 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:23.937632 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:23.937632 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:23.937632 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:23.937632 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:23.937632 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:23.937632 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:23.937632 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:23.937632 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:23.937632 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:23.937632 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:23.937632 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 16:54:23.937632 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 16:54:23.937632 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 16:54:23.938505 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 16:54:23.938505 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:23.938505 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:23.938505 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:23.938505 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:23.938505 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:23.938505 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:23.938505 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:23.938505 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:23.938505 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:23.938505 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:23.938505 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:23.938505 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:23.938505 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:23.938505 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:23.938505 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:23.938505 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:23.939381 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 16:54:23.939381 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 16:54:23.939381 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 16:54:23.939381 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 16:54:23.939381 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 16:54:23.939381 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 16:54:23.939381 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:54:23.939381 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:23.941804 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 260/537] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 16:54:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:23.947909 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2a0c31-2814-48d3-a530-031d27778861 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:23.964387 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793d6d33-e53a-46ad-8533-140dbce9779b None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:24.007395 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319beb13-d138-4845-97fb-2dbbc1102543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:24.008387 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319beb13-d138-4845-97fb-2dbbc1102543 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:24.008868 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319beb13-d138-4845-97fb-2dbbc1102543 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:24.009342 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319beb13-d138-4845-97fb-2dbbc1102543 None admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:24.010225 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319beb13-d138-4845-97fb-2dbbc1102543 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:24.010753 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319beb13-d138-4845-97fb-2dbbc1102543 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:24.012444 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319beb13-d138-4845-97fb-2dbbc1102543 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:24.026717 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 278/538] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 16:54:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:24.033765 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319beb13-d138-4845-97fb-2dbbc1102543 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:24.042683 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:24.083789 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2a0c31-2814-48d3-a530-031d27778861 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:24.123176 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a4c920-0c43-4e7f-b4ec-bcd3aab5a4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:24.124199 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4c920-0c43-4e7f-b4ec-bcd3aab5a4ee None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:24.124583 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4c920-0c43-4e7f-b4ec-bcd3aab5a4ee None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:24.124984 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4c920-0c43-4e7f-b4ec-bcd3aab5a4ee None admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:24.127203 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a4c920-0c43-4e7f-b4ec-bcd3aab5a4ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:24.127203 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a4c920-0c43-4e7f-b4ec-bcd3aab5a4ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:24.127806 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a4c920-0c43-4e7f-b4ec-bcd3aab5a4ee None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:24.131983 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:24.145075 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 261/539] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 16:54:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:24.151252 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a4c920-0c43-4e7f-b4ec-bcd3aab5a4ee None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:24.192378 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319beb13-d138-4845-97fb-2dbbc1102543 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:24.243717 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cfd011-2a84-4f53-90cb-b16f6d49bd39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:24.244343 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cfd011-2a84-4f53-90cb-b16f6d49bd39 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:24.244625 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cfd011-2a84-4f53-90cb-b16f6d49bd39 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:24.244899 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cfd011-2a84-4f53-90cb-b16f6d49bd39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:24.245640 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cfd011-2a84-4f53-90cb-b16f6d49bd39 None admin] RBAC: Adding query filter params (role.id=5e0f402e1af84b44a723ee0730315d74, scope.project.id=2293c0aab61f44849d0d48b214cde7ea, user.id=fe2403ede53148b2a113ffa2a605e987) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:24.245950 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cfd011-2a84-4f53-90cb-b16f6d49bd39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:24.247429 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cfd011-2a84-4f53-90cb-b16f6d49bd39 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:24.259208 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 279/540] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 16:54:24 2022] GET /identity/v3/role_assignments?user.id=fe2403ede53148b2a113ffa2a605e987&scope.project.id=2293c0aab61f44849d0d48b214cde7ea&role.id=5e0f402e1af84b44a723ee0730315d74 => generated 259 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:24.266616 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a4c920-0c43-4e7f-b4ec-bcd3aab5a4ee None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:24.307959 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50e35d6-fb4f-4bfe-91db-e139083ecbd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:24.308952 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e35d6-fb4f-4bfe-91db-e139083ecbd8 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:24.309434 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e35d6-fb4f-4bfe-91db-e139083ecbd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:24.309906 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e35d6-fb4f-4bfe-91db-e139083ecbd8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:24.311054 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50e35d6-fb4f-4bfe-91db-e139083ecbd8 None admin] RBAC: Adding query filter params (role.id=5e0f402e1af84b44a723ee0730315d74, scope.project.id=2293c0aab61f44849d0d48b214cde7ea, user.id=85876a26a22e4c2c882883f7e978e4f1) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:24.311582 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50e35d6-fb4f-4bfe-91db-e139083ecbd8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:24.313219 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50e35d6-fb4f-4bfe-91db-e139083ecbd8 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:24.319987 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 262/541] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 16:54:24 2022] GET /identity/v3/role_assignments?user.id=85876a26a22e4c2c882883f7e978e4f1&scope.project.id=2293c0aab61f44849d0d48b214cde7ea&role.id=5e0f402e1af84b44a723ee0730315d74 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:54:24.582342 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b523171-94d4-433a-a3ab-58df27d58f62 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:24.583230 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b523171-94d4-433a-a3ab-58df27d58f62 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:24.583659 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b523171-94d4-433a-a3ab-58df27d58f62 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:24.585382 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 280/542] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:54:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:54:24.591281 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f464fe-e6de-4c74-8d1d-4d6894091089 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:24.592284 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f464fe-e6de-4c74-8d1d-4d6894091089 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:24.592663 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f464fe-e6de-4c74-8d1d-4d6894091089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:24.615966 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-26f464fe-e6de-4c74-8d1d-4d6894091089 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:54:24.617193 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-26f464fe-e6de-4c74-8d1d-4d6894091089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:54:24.693579 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:24.706246 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 263/543] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:54:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:54:25.068289 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b523171-94d4-433a-a3ab-58df27d58f62 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:25.152612 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:25.199135 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b523171-94d4-433a-a3ab-58df27d58f62 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:25.240857 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abebc021-4796-4cc1-b38c-b4bb9c137c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:25.241649 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abebc021-4796-4cc1-b38c-b4bb9c137c18 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:25.241935 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abebc021-4796-4cc1-b38c-b4bb9c137c18 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:25.242210 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abebc021-4796-4cc1-b38c-b4bb9c137c18 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:25.261985 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abebc021-4796-4cc1-b38c-b4bb9c137c18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:54:25.267259 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-abebc021-4796-4cc1-b38c-b4bb9c137c18 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 16:54:25.267259 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:25.267259 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:25.267259 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:25.267259 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:25.267259 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:25.267259 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:25.267259 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:25.267259 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:25.267259 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:25.267259 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:25.267259 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:25.267259 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:54:25.267259 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:54:25.267259 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:54:25.267259 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:54:25.268097 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:54:25.268097 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:54:25.268097 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:54:25.268097 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:54:25.268097 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:25.268097 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:25.268097 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:25.268097 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:25.268097 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:25.268097 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:25.268097 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:25.268097 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:25.268097 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:25.268097 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:25.268097 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:25.268097 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:25.269319 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:25.269319 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:25.269319 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:25.269319 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:25.269319 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:54:25.269319 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:54:25.269319 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:54:25.269319 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:54:25.269319 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:54:25.269319 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:54:25.269319 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 16:54:25.269319 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:25.269319 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 281/544] 10.222.0.82 () {56 vars in 1102 bytes} [Thu Aug 18 16:54:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:25.275986 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f464fe-e6de-4c74-8d1d-4d6894091089 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:25.296724 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759964a0-f1cf-4506-b86b-e90345b40765 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:25.297291 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759964a0-f1cf-4506-b86b-e90345b40765 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:25.297570 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759964a0-f1cf-4506-b86b-e90345b40765 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:25.299424 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 282/545] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:54:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:54:25.319342 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88350cee-0389-47dd-a39e-7a29042a4c6d None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:25.319578 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88350cee-0389-47dd-a39e-7a29042a4c6d None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:25.319818 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88350cee-0389-47dd-a39e-7a29042a4c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:25.355049 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-88350cee-0389-47dd-a39e-7a29042a4c6d None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:54:25.356242 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-88350cee-0389-47dd-a39e-7a29042a4c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:54:25.418808 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f464fe-e6de-4c74-8d1d-4d6894091089 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:25.468956 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6b6820-0fb8-4003-b980-27e043a16fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:25.470250 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6b6820-0fb8-4003-b980-27e043a16fed None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:25.471137 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6b6820-0fb8-4003-b980-27e043a16fed None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:25.471640 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6b6820-0fb8-4003-b980-27e043a16fed None admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:25.472505 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6b6820-0fb8-4003-b980-27e043a16fed None admin] RBAC: Adding query filter params (name=service) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:25.473007 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6b6820-0fb8-4003-b980-27e043a16fed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:25.475438 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6b6820-0fb8-4003-b980-27e043a16fed None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:25.490349 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 264/546] 10.222.0.82 () {56 vars in 1103 bytes} [Thu Aug 18 16:54:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:25.492131 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6b6820-0fb8-4003-b980-27e043a16fed None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:25.510951 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 283/547] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:54:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 16:54:25.557323 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:25.609096 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6b6820-0fb8-4003-b980-27e043a16fed None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:25.652509 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe6d3e4-03ab-43a4-aad7-d0e6d376ff71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:25.653513 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe6d3e4-03ab-43a4-aad7-d0e6d376ff71 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:25.654695 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe6d3e4-03ab-43a4-aad7-d0e6d376ff71 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:25.655270 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe6d3e4-03ab-43a4-aad7-d0e6d376ff71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:25.667297 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe6d3e4-03ab-43a4-aad7-d0e6d376ff71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:25.669166 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe6d3e4-03ab-43a4-aad7-d0e6d376ff71 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:25.686387 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-5fe6d3e4-03ab-43a4-aad7-d0e6d376ff71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:25.686387 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:25.686387 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:25.686387 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:25.686387 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:25.686387 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:25.686387 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:25.686387 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:25.686387 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:25.686387 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:25.686387 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:25.686387 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:25.686387 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:54:25.686387 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:54:25.686387 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:54:25.686387 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:54:25.687403 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:25.687403 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:25.687403 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:25.687403 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:25.687403 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:25.687403 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:25.687403 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:25.687403 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:25.687403 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:25.687403 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:25.687403 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:25.687403 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:25.687403 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:25.687403 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:25.687403 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:25.687403 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:25.687403 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:54:25.688505 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:54:25.688505 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:25.688505 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:25.694615 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 265/548] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:54:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 16:54:25.696158 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88350cee-0389-47dd-a39e-7a29042a4c6d None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:25.759192 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:25.804309 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88350cee-0389-47dd-a39e-7a29042a4c6d None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:25.842734 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea35336c-6a36-4354-83e5-526dcc7a9e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:25.843387 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea35336c-6a36-4354-83e5-526dcc7a9e3f None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:25.843667 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea35336c-6a36-4354-83e5-526dcc7a9e3f None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:25.843939 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea35336c-6a36-4354-83e5-526dcc7a9e3f None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:25.844586 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea35336c-6a36-4354-83e5-526dcc7a9e3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:25.844887 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea35336c-6a36-4354-83e5-526dcc7a9e3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:25.846265 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea35336c-6a36-4354-83e5-526dcc7a9e3f None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:25.863080 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 284/549] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:54:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:25.864844 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe6d3e4-03ab-43a4-aad7-d0e6d376ff71 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:25.896199 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea35336c-6a36-4354-83e5-526dcc7a9e3f None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:25.982984 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe6d3e4-03ab-43a4-aad7-d0e6d376ff71 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:25.982984 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:26.035729 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9fc6e8-5220-4eb5-a5a5-877413ceadc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:26.036295 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9fc6e8-5220-4eb5-a5a5-877413ceadc9 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:26.036585 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9fc6e8-5220-4eb5-a5a5-877413ceadc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:26.036865 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9fc6e8-5220-4eb5-a5a5-877413ceadc9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:26.045512 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9fc6e8-5220-4eb5-a5a5-877413ceadc9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:26.048156 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9fc6e8-5220-4eb5-a5a5-877413ceadc9 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:26.050109 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea35336c-6a36-4354-83e5-526dcc7a9e3f None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:26.060099 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-0b9fc6e8-5220-4eb5-a5a5-877413ceadc9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 18 16:54:26.060099 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:26.060099 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:26.060099 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:26.060099 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:26.060099 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:26.060099 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:26.060099 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:26.060099 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:26.060099 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:26.060099 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:26.060099 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:26.060099 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 16:54:26.060099 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 16:54:26.060099 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 16:54:26.060099 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 16:54:26.061614 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:26.061614 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:26.061614 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 16:54:26.061614 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:54:26.061614 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 16:54:26.061614 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:54:26.061614 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:26.061614 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:26.061614 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:26.061614 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:26.061614 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:26.061614 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:26.061614 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:26.061614 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:26.061614 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:26.061614 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:26.061614 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:26.061614 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:26.062489 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:26.062489 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:26.062489 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 16:54:26.062489 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 16:54:26.062489 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 16:54:26.062489 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 16:54:26.062489 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 16:54:26.062489 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 16:54:26.062489 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 18 16:54:26.062489 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:26.066700 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 266/550] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 16:54:25 2022] GET /identity/v3/users/swift => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:26.068223 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9fc6e8-5220-4eb5-a5a5-877413ceadc9 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:26.071704 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:26.109667 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2839216-bd40-426d-9daf-70e8db157644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:26.110346 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2839216-bd40-426d-9daf-70e8db157644 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:26.110632 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2839216-bd40-426d-9daf-70e8db157644 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:26.110935 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2839216-bd40-426d-9daf-70e8db157644 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:26.132344 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2839216-bd40-426d-9daf-70e8db157644 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:54:26.137421 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-c2839216-bd40-426d-9daf-70e8db157644 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 16:54:26.137421 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:26.137421 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:26.137421 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:26.137421 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:26.137421 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:26.137421 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:26.137421 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:26.137421 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:26.137421 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:26.137421 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:26.137421 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:26.137421 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:54:26.137421 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:54:26.137421 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:54:26.137421 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:54:26.138488 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:54:26.138488 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:54:26.138488 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:54:26.138488 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:54:26.138488 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:26.138488 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:26.138488 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:26.138488 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:26.138488 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:26.138488 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:26.138488 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:26.138488 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:26.138488 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:26.138488 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:26.138488 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:26.138488 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:26.139310 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:26.139310 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:26.139310 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:26.139310 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:26.139310 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:54:26.139310 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:54:26.139310 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:54:26.139310 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:54:26.139310 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:54:26.139310 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:54:26.139310 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 16:54:26.139310 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:26.147557 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 285/551] 10.222.0.82 () {56 vars in 1102 bytes} [Thu Aug 18 16:54:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 16:54:26.156219 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2839216-bd40-426d-9daf-70e8db157644 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:26.184682 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9fc6e8-5220-4eb5-a5a5-877413ceadc9 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:26.223330 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d694fa4-9a96-4c9e-b9ca-d8891d9910d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:26.224404 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d694fa4-9a96-4c9e-b9ca-d8891d9910d8 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:26.224888 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d694fa4-9a96-4c9e-b9ca-d8891d9910d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:26.225385 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d694fa4-9a96-4c9e-b9ca-d8891d9910d8 None admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:26.226325 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d694fa4-9a96-4c9e-b9ca-d8891d9910d8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:26.226835 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d694fa4-9a96-4c9e-b9ca-d8891d9910d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:26.228609 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d694fa4-9a96-4c9e-b9ca-d8891d9910d8 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:26.251011 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 267/552] 10.222.0.82 () {56 vars in 1135 bytes} [Thu Aug 18 16:54:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:26.254351 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d694fa4-9a96-4c9e-b9ca-d8891d9910d8 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:26.281314 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:26.363640 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d694fa4-9a96-4c9e-b9ca-d8891d9910d8 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:26.401359 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6021b80d-dc9e-41d9-b0c0-4b4b92655954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:26.402332 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021b80d-dc9e-41d9-b0c0-4b4b92655954 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:26.402820 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021b80d-dc9e-41d9-b0c0-4b4b92655954 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:26.403349 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021b80d-dc9e-41d9-b0c0-4b4b92655954 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:26.438035 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2839216-bd40-426d-9daf-70e8db157644 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:26.476988 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6021b80d-dc9e-41d9-b0c0-4b4b92655954 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:26.478823 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6021b80d-dc9e-41d9-b0c0-4b4b92655954 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:26.507284 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:26.531651 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31149ba-73bc-4969-84cd-ab767d74de77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:26.532249 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31149ba-73bc-4969-84cd-ab767d74de77 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:26.532531 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31149ba-73bc-4969-84cd-ab767d74de77 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:26.532802 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31149ba-73bc-4969-84cd-ab767d74de77 None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:26.533467 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31149ba-73bc-4969-84cd-ab767d74de77 None admin] RBAC: Adding query filter params (name=service) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:26.533767 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31149ba-73bc-4969-84cd-ab767d74de77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:26.535167 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31149ba-73bc-4969-84cd-ab767d74de77 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:26.553316 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 286/553] 10.222.0.82 () {56 vars in 1103 bytes} [Thu Aug 18 16:54:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 397 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:26.558387 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31149ba-73bc-4969-84cd-ab767d74de77 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:26.595093 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 268/554] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:54:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 16:54:26.596824 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605272ee-e484-4d75-8f2d-58c8877180cf None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:26.597344 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605272ee-e484-4d75-8f2d-58c8877180cf None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:26.597823 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605272ee-e484-4d75-8f2d-58c8877180cf None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:26.599657 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 269/555] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:54:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:54:26.600623 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605272ee-e484-4d75-8f2d-58c8877180cf None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:26.623569 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:26.671799 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31149ba-73bc-4969-84cd-ab767d74de77 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:26.708542 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347262f1-dd66-4f09-bd14-b4ef57ec7d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:26.709167 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347262f1-dd66-4f09-bd14-b4ef57ec7d8d None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:26.709486 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347262f1-dd66-4f09-bd14-b4ef57ec7d8d None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:26.709761 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347262f1-dd66-4f09-bd14-b4ef57ec7d8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:26.710384 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605272ee-e484-4d75-8f2d-58c8877180cf None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:26.722736 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347262f1-dd66-4f09-bd14-b4ef57ec7d8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:26.724227 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347262f1-dd66-4f09-bd14-b4ef57ec7d8d None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:26.734335 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:26.740228 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-347262f1-dd66-4f09-bd14-b4ef57ec7d8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:26.740228 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:26.740228 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:26.740228 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:26.740228 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:26.740228 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:26.740228 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:26.740228 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:26.740228 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:26.740228 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:26.740228 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:26.740228 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:26.740228 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:54:26.740228 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:54:26.740228 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:54:26.740228 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:54:26.744343 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:26.744343 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:26.744343 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:26.744343 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:26.744343 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:26.744343 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:26.744343 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:26.744343 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:26.744343 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:26.744343 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:26.744343 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:26.744343 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:26.744343 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:26.744343 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:26.744343 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:26.744343 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:26.744343 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:54:26.745242 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:54:26.745242 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:26.745242 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:26.745242 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 287/556] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:54:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 16:54:26.745242 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299894c2-aaa2-4787-8bee-3cf21cc61795 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:26.745242 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299894c2-aaa2-4787-8bee-3cf21cc61795 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:26.745242 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299894c2-aaa2-4787-8bee-3cf21cc61795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:26.749558 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb615b2-84b1-455d-a73a-a4e34d2dce2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:26.749558 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb615b2-84b1-455d-a73a-a4e34d2dce2f None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:26.749558 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb615b2-84b1-455d-a73a-a4e34d2dce2f None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:26.749558 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb615b2-84b1-455d-a73a-a4e34d2dce2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:26.759474 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb615b2-84b1-455d-a73a-a4e34d2dce2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:26.760944 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb615b2-84b1-455d-a73a-a4e34d2dce2f None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:26.761513 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-299894c2-aaa2-4787-8bee-3cf21cc61795 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:54:26.762753 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-299894c2-aaa2-4787-8bee-3cf21cc61795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:54:26.779875 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-6cb615b2-84b1-455d-a73a-a4e34d2dce2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:26.779875 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:26.779875 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:26.779875 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:26.779875 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:26.779875 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:26.779875 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:26.779875 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:26.779875 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:26.779875 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:26.779875 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:26.779875 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:26.779875 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:54:26.779875 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:54:26.779875 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:54:26.779875 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:54:26.781433 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:26.781433 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:26.781433 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:26.781433 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:26.781433 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:26.781433 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:26.781433 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:26.781433 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:26.781433 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:26.781433 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:26.781433 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:26.781433 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:26.781433 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:26.781433 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:26.781433 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:26.781433 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:26.781433 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:54:26.782278 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:54:26.782278 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:26.782278 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:26.782278 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 270/557] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:54:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:26.782278 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec15447e-319d-4791-9b97-f51b63def862 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:26.782549 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec15447e-319d-4791-9b97-f51b63def862 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:26.782819 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec15447e-319d-4791-9b97-f51b63def862 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:26.784111 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 271/558] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:54:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:54:26.784958 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698de06a-06ed-4e83-998a-8d9001e0530d None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:26.785299 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698de06a-06ed-4e83-998a-8d9001e0530d None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:26.785575 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698de06a-06ed-4e83-998a-8d9001e0530d None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:26.790642 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 272/559] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:54:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:54:26.791450 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698de06a-06ed-4e83-998a-8d9001e0530d None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:26.836652 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 288/560] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:54:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:54:26.837545 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299894c2-aaa2-4787-8bee-3cf21cc61795 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:26.862361 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:26.897852 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698de06a-06ed-4e83-998a-8d9001e0530d None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:26.939432 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365109da-a2e2-46e4-9125-3277f0e3f538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:26.939964 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365109da-a2e2-46e4-9125-3277f0e3f538 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:26.940247 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365109da-a2e2-46e4-9125-3277f0e3f538 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:26.940522 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365109da-a2e2-46e4-9125-3277f0e3f538 None admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:26.941188 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365109da-a2e2-46e4-9125-3277f0e3f538 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:26.941497 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365109da-a2e2-46e4-9125-3277f0e3f538 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:26.942875 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365109da-a2e2-46e4-9125-3277f0e3f538 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:26.947741 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299894c2-aaa2-4787-8bee-3cf21cc61795 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:26.957582 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 273/561] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:54:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:26.958791 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86eef43-4c54-4192-b395-cba4373efcb4 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:26.959113 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86eef43-4c54-4192-b395-cba4373efcb4 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:26.959388 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86eef43-4c54-4192-b395-cba4373efcb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:26.984756 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-e86eef43-4c54-4192-b395-cba4373efcb4 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:54:26.985899 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-e86eef43-4c54-4192-b395-cba4373efcb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:54:26.988119 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b98cd36-a080-469a-9dbc-f7e7ebf5e5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:26.988728 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b98cd36-a080-469a-9dbc-f7e7ebf5e5dd None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:26.989006 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b98cd36-a080-469a-9dbc-f7e7ebf5e5dd None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:26.989279 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b98cd36-a080-469a-9dbc-f7e7ebf5e5dd None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:26.989916 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b98cd36-a080-469a-9dbc-f7e7ebf5e5dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:26.990217 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b98cd36-a080-469a-9dbc-f7e7ebf5e5dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:26.991615 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b98cd36-a080-469a-9dbc-f7e7ebf5e5dd None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:27.004043 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 289/562] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:54:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:27.005091 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5001c0e1-41ce-48a4-a08d-b94f4fb15b4d None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:27.005380 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5001c0e1-41ce-48a4-a08d-b94f4fb15b4d None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:27.005654 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5001c0e1-41ce-48a4-a08d-b94f4fb15b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:27.020016 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:27.023387 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-5001c0e1-41ce-48a4-a08d-b94f4fb15b4d None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:54:27.024582 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-5001c0e1-41ce-48a4-a08d-b94f4fb15b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:54:27.063465 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 274/563] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:54:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:54:27.075809 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86eef43-4c54-4192-b395-cba4373efcb4 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:27.093682 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 290/564] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:54:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:54:27.094540 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5001c0e1-41ce-48a4-a08d-b94f4fb15b4d None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:27.120677 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:27.193093 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86eef43-4c54-4192-b395-cba4373efcb4 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:27.231651 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a7d30e-3b30-496e-9fd1-be322c5c388d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:27.233566 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7d30e-3b30-496e-9fd1-be322c5c388d None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:27.234050 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7d30e-3b30-496e-9fd1-be322c5c388d None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:27.234518 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7d30e-3b30-496e-9fd1-be322c5c388d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:27.239625 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5001c0e1-41ce-48a4-a08d-b94f4fb15b4d None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:27.242829 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a7d30e-3b30-496e-9fd1-be322c5c388d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:27.244607 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a7d30e-3b30-496e-9fd1-be322c5c388d None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:27.257996 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-c9a7d30e-3b30-496e-9fd1-be322c5c388d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 18 16:54:27.257996 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:27.257996 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:27.257996 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:27.257996 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:27.257996 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:27.257996 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:27.257996 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:27.257996 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:27.257996 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:27.257996 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:27.257996 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:27.257996 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 16:54:27.257996 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 16:54:27.257996 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 16:54:27.257996 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 16:54:27.259185 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:27.259185 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:27.259185 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 16:54:27.259185 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:54:27.259185 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 16:54:27.259185 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:54:27.259185 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:27.259185 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:27.259185 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:27.259185 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:27.259185 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:27.259185 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:27.259185 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:27.259185 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:27.259185 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:27.259185 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:27.259185 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:27.259185 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:27.260223 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:27.260223 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:27.260223 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 16:54:27.260223 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 16:54:27.260223 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 16:54:27.260223 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 16:54:27.260223 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 16:54:27.260223 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 16:54:27.260223 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 18 16:54:27.260223 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:27.262680 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 275/565] 10.222.0.82 () {56 vars in 1093 bytes} [Thu Aug 18 16:54:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:27.263717 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a7d30e-3b30-496e-9fd1-be322c5c388d None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:27.279704 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad34973-e6f4-4860-abb9-df3da57c01ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:27.280611 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad34973-e6f4-4860-abb9-df3da57c01ac None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:27.280898 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad34973-e6f4-4860-abb9-df3da57c01ac None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:27.281169 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad34973-e6f4-4860-abb9-df3da57c01ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:27.290306 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad34973-e6f4-4860-abb9-df3da57c01ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:27.292117 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad34973-e6f4-4860-abb9-df3da57c01ac None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:27.293114 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:27.308468 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-7ad34973-e6f4-4860-abb9-df3da57c01ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:54:27.308468 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:27.308468 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:27.308468 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:27.308468 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:27.308468 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:27.308468 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:27.308468 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:27.308468 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:27.308468 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:27.308468 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:27.308468 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:27.308468 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 16:54:27.308468 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 16:54:27.308468 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 16:54:27.309372 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 16:54:27.309372 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:27.309372 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:27.309372 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:27.309372 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:27.309372 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:27.309372 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:27.309372 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:27.309372 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:27.309372 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:27.309372 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:27.309372 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:27.309372 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:27.309372 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:27.309372 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:27.309372 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:27.309372 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:27.310268 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 16:54:27.310268 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 16:54:27.310268 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 16:54:27.310268 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 16:54:27.310268 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 16:54:27.310268 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 16:54:27.310268 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:54:27.310268 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:27.310268 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 291/566] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 16:54:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:27.311219 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad34973-e6f4-4860-abb9-df3da57c01ac None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:27.380404 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a7d30e-3b30-496e-9fd1-be322c5c388d None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:27.391963 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:27.421622 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad34973-e6f4-4860-abb9-df3da57c01ac None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:27.438589 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f82e0a-9934-4c2f-a335-c2c636fa5dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:27.439243 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f82e0a-9934-4c2f-a335-c2c636fa5dcd None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:27.439528 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f82e0a-9934-4c2f-a335-c2c636fa5dcd None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:27.439804 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f82e0a-9934-4c2f-a335-c2c636fa5dcd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:27.449715 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f82e0a-9934-4c2f-a335-c2c636fa5dcd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:54:27.454745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-f4f82e0a-9934-4c2f-a335-c2c636fa5dcd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 16:54:27.454745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:27.454745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:27.454745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:27.454745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:27.454745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:27.454745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:27.454745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:27.454745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:27.454745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:27.454745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:27.454745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:27.454745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:54:27.454745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:54:27.454745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:54:27.454745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:54:27.455685 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:54:27.455685 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:54:27.455685 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:54:27.455685 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:54:27.455685 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:27.455685 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:27.455685 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:27.455685 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:27.455685 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:27.455685 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:27.455685 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:27.455685 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:27.455685 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:27.455685 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:27.455685 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:27.455685 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:27.457031 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:27.457031 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:27.457031 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:27.457031 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:27.457031 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:54:27.457031 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:54:27.457031 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:54:27.457031 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:54:27.457031 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:54:27.457031 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:54:27.457031 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 16:54:27.457031 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:27.457031 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 276/567] 10.222.0.82 () {56 vars in 1102 bytes} [Thu Aug 18 16:54:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:27.457031 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f82e0a-9934-4c2f-a335-c2c636fa5dcd None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:27.459579 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546c5682-7796-4f0c-8349-f8ada17b3ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:27.460250 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546c5682-7796-4f0c-8349-f8ada17b3ce0 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:27.460557 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546c5682-7796-4f0c-8349-f8ada17b3ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:27.460827 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546c5682-7796-4f0c-8349-f8ada17b3ce0 None admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:27.461554 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546c5682-7796-4f0c-8349-f8ada17b3ce0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:27.461856 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546c5682-7796-4f0c-8349-f8ada17b3ce0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:27.463302 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546c5682-7796-4f0c-8349-f8ada17b3ce0 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:27.481240 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 292/568] 10.222.0.82 () {56 vars in 1133 bytes} [Thu Aug 18 16:54:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:27.482568 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546c5682-7796-4f0c-8349-f8ada17b3ce0 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:27.542441 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:27.558112 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f82e0a-9934-4c2f-a335-c2c636fa5dcd None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:27.582254 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546c5682-7796-4f0c-8349-f8ada17b3ce0 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:27.592944 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b0e16b-f33c-4232-b93c-e5d480663943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:27.593559 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b0e16b-f33c-4232-b93c-e5d480663943 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:27.593839 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b0e16b-f33c-4232-b93c-e5d480663943 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:27.594173 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b0e16b-f33c-4232-b93c-e5d480663943 None admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:27.594869 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b0e16b-f33c-4232-b93c-e5d480663943 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:27.595221 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b0e16b-f33c-4232-b93c-e5d480663943 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:27.596678 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b0e16b-f33c-4232-b93c-e5d480663943 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:27.608433 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 277/569] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 16:54:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:27.609221 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b0e16b-f33c-4232-b93c-e5d480663943 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:27.616575 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:27.622233 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be66824f-9b67-4e81-b9f7-e91992c6d4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:27.622870 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be66824f-9b67-4e81-b9f7-e91992c6d4fa None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:27.623187 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be66824f-9b67-4e81-b9f7-e91992c6d4fa None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:27.623454 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be66824f-9b67-4e81-b9f7-e91992c6d4fa None admin] PATH_INFO: `/v3/roles/service` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:27.633677 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be66824f-9b67-4e81-b9f7-e91992c6d4fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:54:27.638975 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-be66824f-9b67-4e81-b9f7-e91992c6d4fa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 16:54:27.638975 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:27.638975 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:27.638975 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:27.638975 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:27.638975 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:27.638975 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:27.638975 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:27.638975 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:27.638975 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:27.638975 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:27.638975 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:27.638975 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:54:27.638975 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:54:27.638975 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:54:27.638975 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:54:27.640006 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:54:27.640006 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:54:27.640006 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:54:27.640006 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:54:27.640006 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:27.640006 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:27.640006 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:27.640006 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:27.640006 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:27.640006 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:27.640006 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:27.640006 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:27.640006 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:27.640006 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:27.640006 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:27.640006 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:27.641669 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:27.641669 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:27.641669 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:27.641669 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:27.641669 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:54:27.641669 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:54:27.641669 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:54:27.641669 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:54:27.641669 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:54:27.641669 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:54:27.641669 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 16:54:27.641669 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:27.641669 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 293/570] 10.222.0.82 () {56 vars in 1102 bytes} [Thu Aug 18 16:54:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:27.641669 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be66824f-9b67-4e81-b9f7-e91992c6d4fa None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:27.709937 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b0e16b-f33c-4232-b93c-e5d480663943 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:27.739853 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be66824f-9b67-4e81-b9f7-e91992c6d4fa None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:27.743217 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6849248-b1fa-4be2-a203-b9b1885d7030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:27.743881 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6849248-b1fa-4be2-a203-b9b1885d7030 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:27.744108 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6849248-b1fa-4be2-a203-b9b1885d7030 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:27.744384 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6849248-b1fa-4be2-a203-b9b1885d7030 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:27.754722 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6849248-b1fa-4be2-a203-b9b1885d7030 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:54:27.759436 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-b6849248-b1fa-4be2-a203-b9b1885d7030 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 16:54:27.759436 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:27.759436 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:27.759436 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:27.759436 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:27.759436 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:27.759436 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:27.759436 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:27.759436 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:27.759436 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:27.759436 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:27.759436 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:27.759436 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:54:27.759436 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:54:27.759436 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:54:27.759436 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:54:27.760528 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:54:27.760528 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:54:27.760528 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:54:27.760528 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:54:27.760528 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:27.760528 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:27.760528 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:27.760528 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:27.760528 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:27.760528 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:27.760528 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:27.760528 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:27.760528 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:27.760528 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:27.760528 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:27.760528 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:27.761377 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:27.761377 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:27.761377 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:27.761377 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:27.761377 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:54:27.761377 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:54:27.761377 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:54:27.761377 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:54:27.761377 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:54:27.761377 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:54:27.761377 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 16:54:27.761377 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:27.761377 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 278/571] 10.222.0.82 () {56 vars in 1102 bytes} [Thu Aug 18 16:54:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:27.762043 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6849248-b1fa-4be2-a203-b9b1885d7030 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:27.769804 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:27.780001 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8529c1f-a38e-4e9b-9b17-564b925ae92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:27.784000 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8529c1f-a38e-4e9b-9b17-564b925ae92c None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:27.784000 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8529c1f-a38e-4e9b-9b17-564b925ae92c None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:27.784000 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8529c1f-a38e-4e9b-9b17-564b925ae92c None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:27.784000 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8529c1f-a38e-4e9b-9b17-564b925ae92c None admin] RBAC: Adding query filter params (name=service) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:27.784000 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8529c1f-a38e-4e9b-9b17-564b925ae92c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:27.784000 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8529c1f-a38e-4e9b-9b17-564b925ae92c None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:27.791535 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 294/572] 10.222.0.82 () {56 vars in 1103 bytes} [Thu Aug 18 16:54:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:27.792324 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8529c1f-a38e-4e9b-9b17-564b925ae92c None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:27.847045 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:27.867466 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6849248-b1fa-4be2-a203-b9b1885d7030 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:27.894764 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8529c1f-a38e-4e9b-9b17-564b925ae92c None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:27.900834 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48326c6e-fbd6-4eef-8592-2d2c55d66238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:27.901467 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48326c6e-fbd6-4eef-8592-2d2c55d66238 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:27.901755 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48326c6e-fbd6-4eef-8592-2d2c55d66238 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:27.902031 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48326c6e-fbd6-4eef-8592-2d2c55d66238 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:27.912475 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48326c6e-fbd6-4eef-8592-2d2c55d66238 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:27.913936 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48326c6e-fbd6-4eef-8592-2d2c55d66238 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:27.927749 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d66d3e-eae5-4df1-aea8-dcf8c5af5f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:27.928352 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d66d3e-eae5-4df1-aea8-dcf8c5af5f23 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:27.928634 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d66d3e-eae5-4df1-aea8-dcf8c5af5f23 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:27.928909 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d66d3e-eae5-4df1-aea8-dcf8c5af5f23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:27.929350 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-48326c6e-fbd6-4eef-8592-2d2c55d66238 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:27.929350 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:27.929350 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:27.929350 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:27.929350 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:27.929350 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:27.929350 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:27.929350 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:27.929350 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:27.929350 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:27.929350 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:27.929350 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:27.929350 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:54:27.929350 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:54:27.929350 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:54:27.929350 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:54:27.930382 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:27.930382 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:27.930382 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:27.930382 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:27.930382 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:27.930382 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:27.930382 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:27.930382 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:27.930382 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:27.930382 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:27.930382 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:27.930382 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:27.930382 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:27.930382 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:27.930382 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:27.930382 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:27.930382 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:54:27.931486 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:54:27.931486 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:27.931486 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:27.931652 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d66d3e-eae5-4df1-aea8-dcf8c5af5f23 None admin] RBAC: Adding query filter params (role.id=5e0f402e1af84b44a723ee0730315d74, scope.project.id=2293c0aab61f44849d0d48b214cde7ea, user.id=1381673f81584f4b8084823e3aa99e2a) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:27.931652 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d66d3e-eae5-4df1-aea8-dcf8c5af5f23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:27.931790 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 279/573] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:54:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:27.931790 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48326c6e-fbd6-4eef-8592-2d2c55d66238 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:27.931911 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d66d3e-eae5-4df1-aea8-dcf8c5af5f23 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:27.937135 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 295/574] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 16:54:27 2022] GET /identity/v3/role_assignments?user.id=1381673f81584f4b8084823e3aa99e2a&scope.project.id=2293c0aab61f44849d0d48b214cde7ea&role.id=5e0f402e1af84b44a723ee0730315d74 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:27.937905 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d66d3e-eae5-4df1-aea8-dcf8c5af5f23 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:28.001026 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:28.051204 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48326c6e-fbd6-4eef-8592-2d2c55d66238 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:28.061116 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d66d3e-eae5-4df1-aea8-dcf8c5af5f23 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:28.081681 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:28.089227 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8e83fe-956d-4608-9687-73d9ee02ec9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:28.106434 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e83fe-956d-4608-9687-73d9ee02ec9b None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:28.106434 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e83fe-956d-4608-9687-73d9ee02ec9b None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:28.106434 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e83fe-956d-4608-9687-73d9ee02ec9b None admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:28.106434 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8e83fe-956d-4608-9687-73d9ee02ec9b None admin] RBAC: Adding query filter params (name=service) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:28.106434 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8e83fe-956d-4608-9687-73d9ee02ec9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:28.106434 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8e83fe-956d-4608-9687-73d9ee02ec9b None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:28.107834 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b5f646-79d1-4b93-b7b9-186c328d1304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:28.107927 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 280/575] 10.222.0.82 () {56 vars in 1103 bytes} [Thu Aug 18 16:54:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:28.107995 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b5f646-79d1-4b93-b7b9-186c328d1304 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:28.108061 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8e83fe-956d-4608-9687-73d9ee02ec9b None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:28.108137 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b5f646-79d1-4b93-b7b9-186c328d1304 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:28.108137 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b5f646-79d1-4b93-b7b9-186c328d1304 None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:28.108137 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b5f646-79d1-4b93-b7b9-186c328d1304 None admin] RBAC: Adding query filter params (name=service) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:28.108137 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b5f646-79d1-4b93-b7b9-186c328d1304 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:28.108137 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b5f646-79d1-4b93-b7b9-186c328d1304 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:28.120167 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 296/576] 10.222.0.82 () {56 vars in 1103 bytes} [Thu Aug 18 16:54:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:28.120167 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b5f646-79d1-4b93-b7b9-186c328d1304 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:28.218265 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8e83fe-956d-4608-9687-73d9ee02ec9b None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:28.234562 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b5f646-79d1-4b93-b7b9-186c328d1304 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:28.236377 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:28.254264 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf864ca1-88f9-4edb-b93e-9ae7e385a7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:28.254831 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf864ca1-88f9-4edb-b93e-9ae7e385a7b3 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:28.255146 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf864ca1-88f9-4edb-b93e-9ae7e385a7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:28.255423 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf864ca1-88f9-4edb-b93e-9ae7e385a7b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:28.267089 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf864ca1-88f9-4edb-b93e-9ae7e385a7b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:28.268560 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf864ca1-88f9-4edb-b93e-9ae7e385a7b3 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:28.271390 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024c6b5c-cc32-44c5-aa7c-8eabcd35ede7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:28.272000 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c6b5c-cc32-44c5-aa7c-8eabcd35ede7 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:28.272281 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c6b5c-cc32-44c5-aa7c-8eabcd35ede7 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:28.272554 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c6b5c-cc32-44c5-aa7c-8eabcd35ede7 None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:28.273198 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024c6b5c-cc32-44c5-aa7c-8eabcd35ede7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:28.273505 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024c6b5c-cc32-44c5-aa7c-8eabcd35ede7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:28.275012 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024c6b5c-cc32-44c5-aa7c-8eabcd35ede7 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:28.284660 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-bf864ca1-88f9-4edb-b93e-9ae7e385a7b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:28.284660 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:28.284660 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:28.284660 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:28.284660 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:28.284660 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:28.284660 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:28.284660 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:28.284660 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:28.284660 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:28.284660 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:28.284660 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:28.284660 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:54:28.284660 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:54:28.284660 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:54:28.284660 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:54:28.285588 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:28.285588 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:28.285588 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:28.285588 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:28.285588 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:28.285588 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:28.285588 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:28.285588 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:28.285588 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:28.285588 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:28.285588 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:28.285588 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:28.285588 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:28.285588 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:28.285588 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:28.285588 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:28.285588 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:54:28.296153 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:54:28.296153 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:28.296153 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:28.296153 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 281/577] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:54:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:28.296153 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf864ca1-88f9-4edb-b93e-9ae7e385a7b3 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:28.296454 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 297/578] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:54:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:28.296454 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024c6b5c-cc32-44c5-aa7c-8eabcd35ede7 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:28.331541 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:28.400952 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024c6b5c-cc32-44c5-aa7c-8eabcd35ede7 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:28.401286 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf864ca1-88f9-4edb-b93e-9ae7e385a7b3 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:28.438889 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eceecf5-3e83-4a13-96c9-422ff49e046e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:28.439568 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eceecf5-3e83-4a13-96c9-422ff49e046e None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:28.439851 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eceecf5-3e83-4a13-96c9-422ff49e046e None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:28.440125 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eceecf5-3e83-4a13-96c9-422ff49e046e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:28.441562 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060272d9-3fd6-475b-92f7-8d30370b335e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:28.442420 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060272d9-3fd6-475b-92f7-8d30370b335e None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:28.442925 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060272d9-3fd6-475b-92f7-8d30370b335e None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:28.443420 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060272d9-3fd6-475b-92f7-8d30370b335e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:28.452806 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eceecf5-3e83-4a13-96c9-422ff49e046e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:28.454277 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eceecf5-3e83-4a13-96c9-422ff49e046e None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:28.454702 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060272d9-3fd6-475b-92f7-8d30370b335e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:28.456406 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060272d9-3fd6-475b-92f7-8d30370b335e None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:28.468896 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-0eceecf5-3e83-4a13-96c9-422ff49e046e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:28.468896 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:28.468896 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:28.468896 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:28.468896 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:28.468896 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:28.468896 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:28.468896 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:28.468896 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:28.468896 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:28.468896 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:28.468896 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:28.468896 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:54:28.468896 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:54:28.468896 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:54:28.468896 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:54:28.469987 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:28.469987 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:28.469987 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:28.469987 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:28.469987 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:28.469987 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:28.469987 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:28.469987 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:28.469987 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:28.469987 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:28.469987 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:28.469987 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:28.469987 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:28.469987 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:28.469987 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:28.469987 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:28.469987 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:54:28.470891 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:54:28.470891 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:28.470891 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:28.470891 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 298/579] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:54:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:28.471130 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eceecf5-3e83-4a13-96c9-422ff49e046e None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:28.473818 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-060272d9-3fd6-475b-92f7-8d30370b335e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:28.473818 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:28.473818 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:28.473818 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:28.473818 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:28.473818 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:28.473818 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:28.473818 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:28.473818 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:28.473818 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:28.473818 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:28.473818 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:28.473818 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:54:28.473818 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:54:28.473818 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:54:28.473818 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:54:28.474708 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:28.474708 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:28.474708 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:28.474708 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:28.474708 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:28.474708 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:28.474708 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:28.474708 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:28.474708 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:28.474708 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:28.474708 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:28.474708 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:28.474708 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:28.474708 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:28.474708 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:28.474708 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:28.474708 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:54:28.475643 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:54:28.475643 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:28.475643 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:28.475643 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 282/580] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:54:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:28.476283 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060272d9-3fd6-475b-92f7-8d30370b335e None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:28.505443 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:28.573908 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:28.583345 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060272d9-3fd6-475b-92f7-8d30370b335e None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:28.584348 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eceecf5-3e83-4a13-96c9-422ff49e046e None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:28.620046 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2786d5-f5e6-4d67-9550-cbf886de0ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:28.620682 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2786d5-f5e6-4d67-9550-cbf886de0ce3 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:28.620968 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2786d5-f5e6-4d67-9550-cbf886de0ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:28.621307 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2786d5-f5e6-4d67-9550-cbf886de0ce3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:28.622475 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb60b8a-1d0d-439b-9d1b-720edcae93e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:28.623598 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb60b8a-1d0d-439b-9d1b-720edcae93e8 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:28.623887 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb60b8a-1d0d-439b-9d1b-720edcae93e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:28.624161 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb60b8a-1d0d-439b-9d1b-720edcae93e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:28.624798 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb60b8a-1d0d-439b-9d1b-720edcae93e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:28.625104 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb60b8a-1d0d-439b-9d1b-720edcae93e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:28.626489 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb60b8a-1d0d-439b-9d1b-720edcae93e8 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:28.631595 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2786d5-f5e6-4d67-9550-cbf886de0ce3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:28.633617 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2786d5-f5e6-4d67-9550-cbf886de0ce3 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:28.642519 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 299/581] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:54:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:28.642519 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb60b8a-1d0d-439b-9d1b-720edcae93e8 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:28.647829 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-dc2786d5-f5e6-4d67-9550-cbf886de0ce3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:54:28.647829 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:28.647829 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:28.647829 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:28.647829 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:28.647829 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:28.647829 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:28.647829 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:28.647829 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:28.647829 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:28.647829 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:28.647829 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:28.647829 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 16:54:28.647829 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 16:54:28.647829 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 16:54:28.648691 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 16:54:28.648691 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:28.648691 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:28.648691 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:28.648691 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:28.648691 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:28.648691 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:28.648691 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:28.648691 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:28.648691 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:28.648691 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:28.648691 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:28.648691 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:28.648691 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:28.648691 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:28.648691 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:28.648691 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:28.649533 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 16:54:28.649533 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 16:54:28.649533 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 16:54:28.649533 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 16:54:28.649533 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 16:54:28.649533 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 16:54:28.649533 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:54:28.649533 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:28.649533 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 283/582] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 16:54:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:28.650014 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2786d5-f5e6-4d67-9550-cbf886de0ce3 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:28.738468 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:28.752496 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb60b8a-1d0d-439b-9d1b-720edcae93e8 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:28.760943 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2786d5-f5e6-4d67-9550-cbf886de0ce3 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:28.791219 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dfd609-24b9-44c3-b7fe-7eb15b13fc71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:28.792279 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dfd609-24b9-44c3-b7fe-7eb15b13fc71 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:28.792876 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dfd609-24b9-44c3-b7fe-7eb15b13fc71 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:28.793466 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dfd609-24b9-44c3-b7fe-7eb15b13fc71 None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:28.794319 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dfd609-24b9-44c3-b7fe-7eb15b13fc71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:28.794825 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dfd609-24b9-44c3-b7fe-7eb15b13fc71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:28.796712 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dfd609-24b9-44c3-b7fe-7eb15b13fc71 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:28.798060 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d867db-c6bd-4aa7-82cd-4697a954e036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:28.798675 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d867db-c6bd-4aa7-82cd-4697a954e036 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:28.798990 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d867db-c6bd-4aa7-82cd-4697a954e036 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:28.799274 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d867db-c6bd-4aa7-82cd-4697a954e036 None admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:28.799915 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d867db-c6bd-4aa7-82cd-4697a954e036 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:28.800221 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d867db-c6bd-4aa7-82cd-4697a954e036 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:28.801590 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d867db-c6bd-4aa7-82cd-4697a954e036 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:28.812016 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 300/583] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:54:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:28.814462 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 284/584] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:54:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:28.815514 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d867db-c6bd-4aa7-82cd-4697a954e036 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:28.817268 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dfd609-24b9-44c3-b7fe-7eb15b13fc71 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:28.831584 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:28.921329 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d867db-c6bd-4aa7-82cd-4697a954e036 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:28.939845 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dfd609-24b9-44c3-b7fe-7eb15b13fc71 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:28.959449 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9947e1fe-080a-4002-b9f8-229668d9575c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:28.960067 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9947e1fe-080a-4002-b9f8-229668d9575c None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:28.960349 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9947e1fe-080a-4002-b9f8-229668d9575c None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:28.960624 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9947e1fe-080a-4002-b9f8-229668d9575c None admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:28.961309 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9947e1fe-080a-4002-b9f8-229668d9575c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:28.961614 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9947e1fe-080a-4002-b9f8-229668d9575c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:28.963082 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9947e1fe-080a-4002-b9f8-229668d9575c None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:28.980133 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 285/585] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 16:54:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:28.980133 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9947e1fe-080a-4002-b9f8-229668d9575c None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:28.981284 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724586b6-db53-4472-a5f7-13298e55695b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:28.981963 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724586b6-db53-4472-a5f7-13298e55695b None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:28.982451 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724586b6-db53-4472-a5f7-13298e55695b None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:28.982865 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724586b6-db53-4472-a5f7-13298e55695b None admin] PATH_INFO: `/v3/users/glance` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:28.983287 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:28.990512 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724586b6-db53-4472-a5f7-13298e55695b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:28.991970 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724586b6-db53-4472-a5f7-13298e55695b None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:29.004426 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-724586b6-db53-4472-a5f7-13298e55695b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 18 16:54:29.004426 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:29.004426 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:29.004426 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:29.004426 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:29.004426 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:29.004426 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:29.004426 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:29.004426 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:29.004426 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:29.004426 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:29.004426 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:29.004426 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 16:54:29.004426 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 16:54:29.004426 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 16:54:29.004426 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 16:54:29.005394 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:29.005394 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:29.005394 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 16:54:29.005394 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:54:29.005394 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 16:54:29.005394 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:54:29.005394 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:29.005394 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:29.005394 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:29.005394 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:29.005394 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:29.005394 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:29.005394 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:29.005394 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:29.005394 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:29.005394 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:29.005394 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:29.005394 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:29.006312 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:29.006312 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:29.006312 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 16:54:29.006312 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 16:54:29.006312 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 16:54:29.006312 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 16:54:29.006312 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 16:54:29.006312 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 16:54:29.006312 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 18 16:54:29.006312 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:29.006312 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 301/586] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:54:28 2022] GET /identity/v3/users/glance => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:29.006312 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724586b6-db53-4472-a5f7-13298e55695b None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:29.082710 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:29.097412 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9947e1fe-080a-4002-b9f8-229668d9575c None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:29.115395 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724586b6-db53-4472-a5f7-13298e55695b None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:29.133520 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d995d7f7-2bd9-4dfc-813c-96c0efb75c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:29.134151 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d995d7f7-2bd9-4dfc-813c-96c0efb75c4f None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:29.134434 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d995d7f7-2bd9-4dfc-813c-96c0efb75c4f None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:29.134710 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d995d7f7-2bd9-4dfc-813c-96c0efb75c4f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:29.142817 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d995d7f7-2bd9-4dfc-813c-96c0efb75c4f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:29.144295 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d995d7f7-2bd9-4dfc-813c-96c0efb75c4f None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:29.149907 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3cbd01-a19a-4727-b716-b0362e6b505a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:29.150638 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3cbd01-a19a-4727-b716-b0362e6b505a None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:29.151072 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3cbd01-a19a-4727-b716-b0362e6b505a None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:29.151448 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3cbd01-a19a-4727-b716-b0362e6b505a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:29.156293 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-d995d7f7-2bd9-4dfc-813c-96c0efb75c4f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 18 16:54:29.156293 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:29.156293 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:29.156293 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:29.156293 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:29.156293 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:29.156293 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:29.156293 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:29.156293 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:29.156293 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:29.156293 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:29.156293 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:29.156293 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 16:54:29.156293 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 16:54:29.156293 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 16:54:29.156293 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 16:54:29.157364 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:29.157364 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:29.157364 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 16:54:29.157364 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:54:29.157364 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 16:54:29.157364 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:54:29.157364 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:29.157364 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:29.157364 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:29.157364 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:29.157364 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:29.157364 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:29.157364 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:29.157364 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:29.157364 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:29.157364 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:29.157364 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:29.157364 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:29.158281 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:29.158281 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:29.158281 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 16:54:29.158281 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 16:54:29.158281 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 16:54:29.158281 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 16:54:29.158281 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 16:54:29.158281 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 16:54:29.158281 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 18 16:54:29.158281 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:29.158281 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 286/587] 10.222.0.82 () {56 vars in 1102 bytes} [Thu Aug 18 16:54:28 2022] GET /identity/v3/users/neutron => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:29.158281 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d995d7f7-2bd9-4dfc-813c-96c0efb75c4f None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:29.166112 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3cbd01-a19a-4727-b716-b0362e6b505a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:29.167718 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3cbd01-a19a-4727-b716-b0362e6b505a None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:29.180019 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-df3cbd01-a19a-4727-b716-b0362e6b505a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 18 16:54:29.180019 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:29.180019 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:29.180019 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:29.180019 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:29.180019 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:29.180019 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:29.180019 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:29.180019 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:29.180019 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:29.180019 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:29.180019 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:29.180019 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 16:54:29.180019 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 16:54:29.180019 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 16:54:29.180019 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 16:54:29.181189 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:29.181189 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:29.181189 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 16:54:29.181189 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:54:29.181189 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 16:54:29.181189 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:54:29.181189 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:29.181189 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:29.181189 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:29.181189 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:29.181189 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:29.181189 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:29.181189 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:29.181189 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:29.181189 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:29.181189 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:29.181189 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:29.181189 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:29.182124 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:29.182124 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:29.182124 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 16:54:29.182124 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 16:54:29.182124 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 16:54:29.182124 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 16:54:29.182124 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 16:54:29.182124 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 16:54:29.182124 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 18 16:54:29.182124 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:29.182124 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 302/588] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:54:29 2022] GET /identity/v3/users/cinder => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:29.182934 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3cbd01-a19a-4727-b716-b0362e6b505a None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:29.227369 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:29.268728 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d995d7f7-2bd9-4dfc-813c-96c0efb75c4f None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:29.287990 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3cbd01-a19a-4727-b716-b0362e6b505a None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:29.303030 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f51f7fd-aefe-4701-9a13-299629ee56a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:29.303772 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f51f7fd-aefe-4701-9a13-299629ee56a7 None admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:29.304133 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f51f7fd-aefe-4701-9a13-299629ee56a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:29.304474 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f51f7fd-aefe-4701-9a13-299629ee56a7 None admin] PATH_INFO: `/v3/projects/2293c0aab61f44849d0d48b214cde7ea/users/55723c756cae4e7ab636054db48de02d/roles/5e0f402e1af84b44a723ee0730315d74` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:29.317611 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:29.322908 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f51f7fd-aefe-4701-9a13-299629ee56a7 None admin] RBAC: Authorizing `identity:create_grant(project_id=2293c0aab61f44849d0d48b214cde7ea, user_id=55723c756cae4e7ab636054db48de02d, role_id=5e0f402e1af84b44a723ee0730315d74)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:29.327502 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef44acef-e782-4708-b767-b82051327102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:29.327629 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f51f7fd-aefe-4701-9a13-299629ee56a7 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:29.327720 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef44acef-e782-4708-b767-b82051327102 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:29.327720 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef44acef-e782-4708-b767-b82051327102 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:29.327720 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef44acef-e782-4708-b767-b82051327102 None admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:29.327720 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef44acef-e782-4708-b767-b82051327102 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:29.327720 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef44acef-e782-4708-b767-b82051327102 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:29.328259 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef44acef-e782-4708-b767-b82051327102 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:29.347272 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 303/589] 10.222.0.82 () {56 vars in 1137 bytes} [Thu Aug 18 16:54:29 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:29.347995 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef44acef-e782-4708-b767-b82051327102 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:29.413687 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 287/590] 10.222.0.82 () {58 vars in 1439 bytes} [Thu Aug 18 16:54:29 2022] PUT /identity/v3/projects/2293c0aab61f44849d0d48b214cde7ea/users/55723c756cae4e7ab636054db48de02d/roles/5e0f402e1af84b44a723ee0730315d74 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 16:54:29.414942 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f51f7fd-aefe-4701-9a13-299629ee56a7 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:29.471773 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef44acef-e782-4708-b767-b82051327102 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:29.519029 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c8af12-af4d-4af8-a716-72c43eb393e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:29.520169 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c8af12-af4d-4af8-a716-72c43eb393e2 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:29.520634 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c8af12-af4d-4af8-a716-72c43eb393e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:29.521205 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c8af12-af4d-4af8-a716-72c43eb393e2 None admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:29.522310 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c8af12-af4d-4af8-a716-72c43eb393e2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:29.522961 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c8af12-af4d-4af8-a716-72c43eb393e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:29.524644 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c8af12-af4d-4af8-a716-72c43eb393e2 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:29.542840 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f51f7fd-aefe-4701-9a13-299629ee56a7 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:29.549027 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 304/591] 10.222.0.82 () {56 vars in 1139 bytes} [Thu Aug 18 16:54:29 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:29.550206 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c8af12-af4d-4af8-a716-72c43eb393e2 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:29.564203 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:29.590074 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a224f8a4-1d69-490a-b1ad-b838f3c2e42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:29.591621 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a224f8a4-1d69-490a-b1ad-b838f3c2e42c None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:29.591907 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a224f8a4-1d69-490a-b1ad-b838f3c2e42c None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:29.592178 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a224f8a4-1d69-490a-b1ad-b838f3c2e42c None admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:29.592919 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a224f8a4-1d69-490a-b1ad-b838f3c2e42c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:29.593218 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a224f8a4-1d69-490a-b1ad-b838f3c2e42c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:29.594678 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a224f8a4-1d69-490a-b1ad-b838f3c2e42c None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:29.611348 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:29.617349 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 288/592] 10.222.0.82 () {56 vars in 1137 bytes} [Thu Aug 18 16:54:29 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:29.618164 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a224f8a4-1d69-490a-b1ad-b838f3c2e42c None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:29.686511 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c8af12-af4d-4af8-a716-72c43eb393e2 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:29.733644 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a224f8a4-1d69-490a-b1ad-b838f3c2e42c None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:29.737730 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0837bed-7167-4981-8322-4fc9e510b585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:29.738490 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0837bed-7167-4981-8322-4fc9e510b585 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:29.738847 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0837bed-7167-4981-8322-4fc9e510b585 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:29.739210 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0837bed-7167-4981-8322-4fc9e510b585 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:29.753849 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0837bed-7167-4981-8322-4fc9e510b585 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:29.755429 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0837bed-7167-4981-8322-4fc9e510b585 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:29.774347 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-b0837bed-7167-4981-8322-4fc9e510b585 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:29.774347 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:29.774347 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:29.774347 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:29.774347 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:29.774347 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:29.774347 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:29.774347 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:29.774347 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:29.774347 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:29.774347 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:29.774347 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:29.774347 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:54:29.774347 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:54:29.774347 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:54:29.774347 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:54:29.775453 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:29.775453 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:29.775453 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:29.775453 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:29.775453 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:29.775453 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:29.775453 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:29.775453 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:29.775453 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:29.775453 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:29.775453 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:29.775453 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:29.775453 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:29.775453 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:29.775453 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:29.775453 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:29.775453 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:54:29.776316 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:54:29.776316 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:29.776316 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:29.778195 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 305/593] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:54:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:29.780032 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0837bed-7167-4981-8322-4fc9e510b585 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:29.783815 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ce3dd2-7115-4d2a-9e1e-a88eecd80aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:29.785403 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce3dd2-7115-4d2a-9e1e-a88eecd80aad None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:29.785917 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce3dd2-7115-4d2a-9e1e-a88eecd80aad None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:29.788334 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce3dd2-7115-4d2a-9e1e-a88eecd80aad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:29.800619 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ce3dd2-7115-4d2a-9e1e-a88eecd80aad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:29.802419 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ce3dd2-7115-4d2a-9e1e-a88eecd80aad None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:29.818385 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-30ce3dd2-7115-4d2a-9e1e-a88eecd80aad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:29.818385 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:29.818385 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:29.818385 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:29.818385 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:29.818385 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:29.818385 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:29.818385 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:29.818385 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:29.818385 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:29.818385 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:29.818385 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:29.818385 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:54:29.818385 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:54:29.818385 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:54:29.818385 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:54:29.819364 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:29.819364 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:29.819364 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:29.819364 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:29.819364 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:29.819364 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:29.819364 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:29.819364 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:29.819364 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:29.819364 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:29.819364 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:29.819364 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:29.819364 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:29.819364 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:29.819364 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:29.819364 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:29.819364 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:54:29.820264 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:54:29.820264 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:29.820264 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:29.822452 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 289/594] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:54:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:29.828638 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ce3dd2-7115-4d2a-9e1e-a88eecd80aad None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:29.856214 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:29.904251 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:29.907634 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0837bed-7167-4981-8322-4fc9e510b585 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:29.942612 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ce3dd2-7115-4d2a-9e1e-a88eecd80aad None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:29.945952 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f4595d-32df-411d-8455-0a041ed77898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:29.946581 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f4595d-32df-411d-8455-0a041ed77898 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:29.946859 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f4595d-32df-411d-8455-0a041ed77898 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:29.947159 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f4595d-32df-411d-8455-0a041ed77898 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:29.957848 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f4595d-32df-411d-8455-0a041ed77898 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:29.959351 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f4595d-32df-411d-8455-0a041ed77898 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:29.975214 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-52f4595d-32df-411d-8455-0a041ed77898 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:29.975214 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:29.975214 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:29.975214 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:29.975214 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:29.975214 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:29.975214 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:29.975214 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:29.975214 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:29.975214 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:29.975214 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:29.975214 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:29.975214 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:54:29.975214 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:54:29.975214 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:54:29.975214 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:54:29.976159 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:29.976159 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:29.976159 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:29.976159 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:29.976159 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:29.976159 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:29.976159 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:29.976159 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:29.976159 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:29.976159 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:29.976159 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:29.976159 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:29.976159 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:29.976159 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:29.976159 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:29.976159 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:29.976159 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:54:29.977023 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:54:29.977023 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:29.977023 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:29.977023 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 306/595] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:54:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:29.977253 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f4595d-32df-411d-8455-0a041ed77898 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:29.984036 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5da93c2-3e55-4fba-abf2-0e56f7fad63d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:29.984890 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5da93c2-3e55-4fba-abf2-0e56f7fad63d None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:29.985178 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5da93c2-3e55-4fba-abf2-0e56f7fad63d None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:29.985455 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5da93c2-3e55-4fba-abf2-0e56f7fad63d None admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:29.986103 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5da93c2-3e55-4fba-abf2-0e56f7fad63d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:29.986411 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5da93c2-3e55-4fba-abf2-0e56f7fad63d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:29.987860 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5da93c2-3e55-4fba-abf2-0e56f7fad63d None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:30.001430 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 290/596] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:54:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:30.002660 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f327da13-eea6-43d7-a97b-83d4618c04b6 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:30.002968 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f327da13-eea6-43d7-a97b-83d4618c04b6 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:30.003249 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f327da13-eea6-43d7-a97b-83d4618c04b6 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:30.007070 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 291/597] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:54:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:54:30.007070 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f327da13-eea6-43d7-a97b-83d4618c04b6 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:30.093960 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f4595d-32df-411d-8455-0a041ed77898 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:30.095977 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:30.115835 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f327da13-eea6-43d7-a97b-83d4618c04b6 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:30.129417 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8152672-0885-494e-8032-69401e6ff8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:30.130185 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8152672-0885-494e-8032-69401e6ff8f7 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:30.130473 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8152672-0885-494e-8032-69401e6ff8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:30.131159 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8152672-0885-494e-8032-69401e6ff8f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:30.131801 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8152672-0885-494e-8032-69401e6ff8f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:30.132109 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8152672-0885-494e-8032-69401e6ff8f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:30.133481 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8152672-0885-494e-8032-69401e6ff8f7 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:30.146419 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 307/598] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:54:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:30.147065 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8152672-0885-494e-8032-69401e6ff8f7 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:30.155047 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58575ed8-0b4a-4909-ac28-ddd55405bf09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:30.155589 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58575ed8-0b4a-4909-ac28-ddd55405bf09 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:30.155870 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58575ed8-0b4a-4909-ac28-ddd55405bf09 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:30.156154 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58575ed8-0b4a-4909-ac28-ddd55405bf09 None admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:30.156796 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58575ed8-0b4a-4909-ac28-ddd55405bf09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:30.157104 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58575ed8-0b4a-4909-ac28-ddd55405bf09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:30.158505 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58575ed8-0b4a-4909-ac28-ddd55405bf09 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:30.160250 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:30.170682 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 292/599] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:54:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:30.171819 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73647672-a0c8-49a5-a592-d7cf0f530509 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:30.172111 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73647672-a0c8-49a5-a592-d7cf0f530509 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:30.172527 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73647672-a0c8-49a5-a592-d7cf0f530509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:30.190564 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-73647672-a0c8-49a5-a592-d7cf0f530509 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:54:30.191791 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-73647672-a0c8-49a5-a592-d7cf0f530509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:54:30.254818 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8152672-0885-494e-8032-69401e6ff8f7 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:30.264885 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 293/600] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:54:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:54:30.266054 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73647672-a0c8-49a5-a592-d7cf0f530509 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:30.299385 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760ab00d-2d62-466b-a323-ce2b15bdce51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:30.300325 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ab00d-2d62-466b-a323-ce2b15bdce51 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:30.300786 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ab00d-2d62-466b-a323-ce2b15bdce51 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:30.301233 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ab00d-2d62-466b-a323-ce2b15bdce51 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:30.308112 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760ab00d-2d62-466b-a323-ce2b15bdce51 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:30.310294 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760ab00d-2d62-466b-a323-ce2b15bdce51 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:30.321574 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-760ab00d-2d62-466b-a323-ce2b15bdce51 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:54:30.321574 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:30.321574 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:30.321574 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:30.321574 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:30.321574 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:30.321574 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:30.321574 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:30.321574 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:30.321574 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:30.321574 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:30.321574 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:30.321574 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 16:54:30.321574 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 16:54:30.321574 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 16:54:30.322436 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 16:54:30.322436 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:30.322436 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:30.322436 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:30.322436 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:30.322436 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:30.322436 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:30.322436 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:30.322436 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:30.322436 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:30.322436 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:30.322436 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:30.322436 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:30.322436 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:30.322436 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:30.322436 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:30.322436 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:30.323321 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 16:54:30.323321 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 16:54:30.323321 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 16:54:30.323321 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 16:54:30.323321 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 16:54:30.323321 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 16:54:30.323321 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:54:30.323321 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:30.330703 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 308/601] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 16:54:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:30.331737 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760ab00d-2d62-466b-a323-ce2b15bdce51 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:30.359326 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:30.384185 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73647672-a0c8-49a5-a592-d7cf0f530509 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:30.404301 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:30.421139 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aad012-2e3a-4f36-8d0b-60029e650c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:30.421842 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aad012-2e3a-4f36-8d0b-60029e650c17 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:30.422193 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aad012-2e3a-4f36-8d0b-60029e650c17 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:30.422564 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aad012-2e3a-4f36-8d0b-60029e650c17 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:30.430698 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aad012-2e3a-4f36-8d0b-60029e650c17 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:30.432127 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aad012-2e3a-4f36-8d0b-60029e650c17 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:30.437130 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760ab00d-2d62-466b-a323-ce2b15bdce51 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:30.445373 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-d5aad012-2e3a-4f36-8d0b-60029e650c17 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:54:30.445373 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:30.445373 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:30.445373 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:30.445373 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:30.445373 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:30.445373 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:30.445373 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:30.445373 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:30.445373 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:30.445373 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:30.445373 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:30.445373 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 16:54:30.445373 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 16:54:30.445373 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 16:54:30.446441 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 16:54:30.446441 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:30.446441 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:30.446441 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:30.446441 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:30.446441 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:30.446441 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:30.446441 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:30.446441 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:30.446441 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:30.446441 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:30.446441 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:30.446441 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:30.446441 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:30.446441 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:30.446441 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:30.446441 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:30.447541 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 16:54:30.447541 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 16:54:30.447541 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 16:54:30.447541 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 16:54:30.447541 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 16:54:30.447541 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 16:54:30.447541 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:54:30.447541 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:30.448794 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 294/602] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 16:54:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:30.449747 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aad012-2e3a-4f36-8d0b-60029e650c17 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:30.478984 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e439010-51c8-4a69-a529-1c9dde3d5b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:30.480151 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e439010-51c8-4a69-a529-1c9dde3d5b58 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:30.481033 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e439010-51c8-4a69-a529-1c9dde3d5b58 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:30.481456 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e439010-51c8-4a69-a529-1c9dde3d5b58 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:30.492492 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e439010-51c8-4a69-a529-1c9dde3d5b58 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:30.494144 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e439010-51c8-4a69-a529-1c9dde3d5b58 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:30.509809 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-8e439010-51c8-4a69-a529-1c9dde3d5b58 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:54:30.509809 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:30.509809 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:30.509809 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:30.509809 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:30.509809 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:30.509809 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:30.509809 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:30.509809 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:30.509809 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:30.509809 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:30.509809 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:30.509809 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 16:54:30.509809 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 16:54:30.509809 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 16:54:30.510887 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 16:54:30.510887 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:30.510887 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:30.510887 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:30.510887 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:30.510887 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:30.510887 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:30.510887 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:30.510887 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:30.510887 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:30.510887 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:30.510887 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:30.510887 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:30.510887 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:30.510887 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:30.510887 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:30.510887 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:30.511986 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 16:54:30.511986 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 16:54:30.511986 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 16:54:30.511986 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 16:54:30.511986 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 16:54:30.511986 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 16:54:30.511986 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:54:30.511986 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:30.519515 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 309/603] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 16:54:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:30.519515 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e439010-51c8-4a69-a529-1c9dde3d5b58 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:30.583827 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aad012-2e3a-4f36-8d0b-60029e650c17 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:30.615789 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:30.620602 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff7eb3d-fbee-4a34-b550-2654d2b7872a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:30.621218 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7eb3d-fbee-4a34-b550-2654d2b7872a None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:30.621504 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7eb3d-fbee-4a34-b550-2654d2b7872a None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:30.621783 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7eb3d-fbee-4a34-b550-2654d2b7872a None admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:30.623330 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff7eb3d-fbee-4a34-b550-2654d2b7872a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:30.623893 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff7eb3d-fbee-4a34-b550-2654d2b7872a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:30.625465 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff7eb3d-fbee-4a34-b550-2654d2b7872a None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:30.632949 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e439010-51c8-4a69-a529-1c9dde3d5b58 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:30.641174 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 295/604] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 16:54:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:30.641174 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff7eb3d-fbee-4a34-b550-2654d2b7872a None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:30.664735 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:30.671110 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24690ac0-2f86-4b59-a26c-bf6bf265e321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:30.675629 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24690ac0-2f86-4b59-a26c-bf6bf265e321 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:30.675629 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24690ac0-2f86-4b59-a26c-bf6bf265e321 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:30.675629 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24690ac0-2f86-4b59-a26c-bf6bf265e321 None admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:30.675629 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24690ac0-2f86-4b59-a26c-bf6bf265e321 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:30.675629 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24690ac0-2f86-4b59-a26c-bf6bf265e321 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:30.675629 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24690ac0-2f86-4b59-a26c-bf6bf265e321 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:30.687279 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 310/605] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 16:54:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:30.687972 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24690ac0-2f86-4b59-a26c-bf6bf265e321 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:30.750103 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff7eb3d-fbee-4a34-b550-2654d2b7872a None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:30.788784 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d76cc2b-8c6d-48fd-ae45-0145fb1c927d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:30.789418 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d76cc2b-8c6d-48fd-ae45-0145fb1c927d None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:30.789697 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d76cc2b-8c6d-48fd-ae45-0145fb1c927d None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:30.789976 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d76cc2b-8c6d-48fd-ae45-0145fb1c927d None admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:30.790676 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d76cc2b-8c6d-48fd-ae45-0145fb1c927d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:30.791012 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d76cc2b-8c6d-48fd-ae45-0145fb1c927d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:30.792459 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d76cc2b-8c6d-48fd-ae45-0145fb1c927d None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:30.796839 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24690ac0-2f86-4b59-a26c-bf6bf265e321 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:30.809801 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 296/606] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 16:54:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:30.809801 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d76cc2b-8c6d-48fd-ae45-0145fb1c927d None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:30.843994 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8268af-6540-481d-9b04-aee2fe1f0bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:30.844637 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8268af-6540-481d-9b04-aee2fe1f0bd9 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:30.844923 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8268af-6540-481d-9b04-aee2fe1f0bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:30.845228 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8268af-6540-481d-9b04-aee2fe1f0bd9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:30.857311 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8268af-6540-481d-9b04-aee2fe1f0bd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:54:30.861247 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:30.862113 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-7d8268af-6540-481d-9b04-aee2fe1f0bd9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 16:54:30.862113 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:30.862113 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:30.862113 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:30.862113 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:30.862113 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:30.862113 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:30.862113 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:30.862113 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:30.862113 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:30.862113 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:30.862113 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:30.862113 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:54:30.862113 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:54:30.862113 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:54:30.862113 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:54:30.863270 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:54:30.863270 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:54:30.863270 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:54:30.863270 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:54:30.863270 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:30.863270 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:30.863270 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:30.863270 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:30.863270 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:30.863270 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:30.863270 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:30.863270 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:30.863270 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:30.863270 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:30.863270 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:30.863270 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:30.864538 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:30.864538 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:30.864538 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:30.864538 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:30.864538 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:54:30.864538 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:54:30.864538 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:54:30.864538 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:54:30.864538 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:54:30.864538 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:54:30.864538 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 16:54:30.864538 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:30.864538 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 311/607] 10.222.0.82 () {56 vars in 1102 bytes} [Thu Aug 18 16:54:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:30.864538 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8268af-6540-481d-9b04-aee2fe1f0bd9 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:30.917628 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d76cc2b-8c6d-48fd-ae45-0145fb1c927d None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:30.920929 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:30.954231 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04eeb25-7409-4141-a76a-3e74af374405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:30.955087 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04eeb25-7409-4141-a76a-3e74af374405 None admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:30.955366 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04eeb25-7409-4141-a76a-3e74af374405 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:30.955706 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04eeb25-7409-4141-a76a-3e74af374405 None admin] PATH_INFO: `/v3/projects/2293c0aab61f44849d0d48b214cde7ea/users/2bee1ba6bb324de4915904f83349b3c4/roles/5e0f402e1af84b44a723ee0730315d74` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:30.972316 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04eeb25-7409-4141-a76a-3e74af374405 None admin] RBAC: Authorizing `identity:create_grant(project_id=2293c0aab61f44849d0d48b214cde7ea, user_id=2bee1ba6bb324de4915904f83349b3c4, role_id=5e0f402e1af84b44a723ee0730315d74)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:30.973674 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8268af-6540-481d-9b04-aee2fe1f0bd9 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:30.973918 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04eeb25-7409-4141-a76a-3e74af374405 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:31.008124 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e49bae-37e6-4ae4-8a9e-9e6c6b811094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:31.008774 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e49bae-37e6-4ae4-8a9e-9e6c6b811094 None admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:31.009054 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e49bae-37e6-4ae4-8a9e-9e6c6b811094 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:31.009328 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e49bae-37e6-4ae4-8a9e-9e6c6b811094 None admin] PATH_INFO: `/v3/projects/2293c0aab61f44849d0d48b214cde7ea/users/fe2403ede53148b2a113ffa2a605e987/roles/5e0f402e1af84b44a723ee0730315d74` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:31.024454 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e49bae-37e6-4ae4-8a9e-9e6c6b811094 None admin] RBAC: Authorizing `identity:create_grant(project_id=2293c0aab61f44849d0d48b214cde7ea, user_id=fe2403ede53148b2a113ffa2a605e987, role_id=5e0f402e1af84b44a723ee0730315d74)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:31.025889 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e49bae-37e6-4ae4-8a9e-9e6c6b811094 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:31.042202 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 297/608] 10.222.0.82 () {58 vars in 1439 bytes} [Thu Aug 18 16:54:30 2022] PUT /identity/v3/projects/2293c0aab61f44849d0d48b214cde7ea/users/2bee1ba6bb324de4915904f83349b3c4/roles/5e0f402e1af84b44a723ee0730315d74 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 16:54:31.043032 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04eeb25-7409-4141-a76a-3e74af374405 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:31.130514 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 312/609] 10.222.0.82 () {58 vars in 1439 bytes} [Thu Aug 18 16:54:30 2022] PUT /identity/v3/projects/2293c0aab61f44849d0d48b214cde7ea/users/fe2403ede53148b2a113ffa2a605e987/roles/5e0f402e1af84b44a723ee0730315d74 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 16:54:31.131707 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e49bae-37e6-4ae4-8a9e-9e6c6b811094 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:31.170495 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04eeb25-7409-4141-a76a-3e74af374405 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:31.190585 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:31.235569 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a996f4b0-3b56-4808-b68f-cb2f1143c918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:31.236176 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a996f4b0-3b56-4808-b68f-cb2f1143c918 None admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:31.236455 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a996f4b0-3b56-4808-b68f-cb2f1143c918 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:31.236733 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a996f4b0-3b56-4808-b68f-cb2f1143c918 None admin] PATH_INFO: `/v3/projects/2293c0aab61f44849d0d48b214cde7ea/users/85876a26a22e4c2c882883f7e978e4f1/roles/5e0f402e1af84b44a723ee0730315d74` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:31.257124 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a996f4b0-3b56-4808-b68f-cb2f1143c918 None admin] RBAC: Authorizing `identity:create_grant(project_id=2293c0aab61f44849d0d48b214cde7ea, user_id=85876a26a22e4c2c882883f7e978e4f1, role_id=5e0f402e1af84b44a723ee0730315d74)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:31.258620 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a996f4b0-3b56-4808-b68f-cb2f1143c918 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:31.307482 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:31.310780 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e49bae-37e6-4ae4-8a9e-9e6c6b811094 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:31.331567 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 298/610] 10.222.0.82 () {58 vars in 1439 bytes} [Thu Aug 18 16:54:31 2022] PUT /identity/v3/projects/2293c0aab61f44849d0d48b214cde7ea/users/85876a26a22e4c2c882883f7e978e4f1/roles/5e0f402e1af84b44a723ee0730315d74 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 16:54:31.360246 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74519e7d-6438-42bf-98fb-9aadcad6fa4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:31.363529 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74519e7d-6438-42bf-98fb-9aadcad6fa4e None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:31.363529 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74519e7d-6438-42bf-98fb-9aadcad6fa4e None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:31.363529 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74519e7d-6438-42bf-98fb-9aadcad6fa4e None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:31.363529 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74519e7d-6438-42bf-98fb-9aadcad6fa4e None admin] RBAC: Adding query filter params (name=service) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:31.363529 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74519e7d-6438-42bf-98fb-9aadcad6fa4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:31.364620 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74519e7d-6438-42bf-98fb-9aadcad6fa4e None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:31.375591 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 313/611] 10.222.0.82 () {56 vars in 1103 bytes} [Thu Aug 18 16:54:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:31.382188 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a996f4b0-3b56-4808-b68f-cb2f1143c918 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:31.452382 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b81fc-2f22-4cd2-8cee-563634ed8ef4 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:31.452722 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b81fc-2f22-4cd2-8cee-563634ed8ef4 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:31.452924 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b81fc-2f22-4cd2-8cee-563634ed8ef4 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:31.454479 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 314/612] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:54:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:54:31.460448 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c43599c-2e6e-4ef2-9b61-7a0feca1636b None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:31.461069 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c43599c-2e6e-4ef2-9b61-7a0feca1636b None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:31.461597 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c43599c-2e6e-4ef2-9b61-7a0feca1636b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:31.482337 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-0c43599c-2e6e-4ef2-9b61-7a0feca1636b None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:54:31.484021 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-0c43599c-2e6e-4ef2-9b61-7a0feca1636b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:54:31.509218 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a996f4b0-3b56-4808-b68f-cb2f1143c918 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:31.552516 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdcb168-917c-4f11-b817-532cd1335f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:31.553297 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdcb168-917c-4f11-b817-532cd1335f20 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:31.553579 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdcb168-917c-4f11-b817-532cd1335f20 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:31.553896 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdcb168-917c-4f11-b817-532cd1335f20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:31.574473 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdcb168-917c-4f11-b817-532cd1335f20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:31.574473 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdcb168-917c-4f11-b817-532cd1335f20 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:31.588701 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 315/613] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:54:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 16:54:31.590640 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-5bdcb168-917c-4f11-b817-532cd1335f20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:31.590640 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:31.590640 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:31.590640 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:31.590640 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:31.590640 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:31.590640 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:31.590640 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:31.590640 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:31.590640 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:31.590640 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:31.590640 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:31.590640 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:54:31.590640 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:54:31.590640 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:54:31.590640 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:54:31.592073 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:31.592073 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:31.592073 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:31.592073 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:31.592073 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:31.592073 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:31.592073 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:31.592073 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:31.592073 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:31.592073 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:31.592073 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:31.592073 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:31.592073 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:31.592073 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:31.592073 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:31.592073 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:31.592073 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:54:31.593142 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:54:31.593142 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:31.593142 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:31.593142 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 299/614] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:54:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:31.597385 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c43599c-2e6e-4ef2-9b61-7a0feca1636b None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:31.698686 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:31.707413 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c43599c-2e6e-4ef2-9b61-7a0feca1636b None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:31.743740 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551e5ff3-30f3-4d95-b76f-f6d8599a75ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:31.744457 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e5ff3-30f3-4d95-b76f-f6d8599a75ad None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:31.744810 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e5ff3-30f3-4d95-b76f-f6d8599a75ad None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:31.745156 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e5ff3-30f3-4d95-b76f-f6d8599a75ad None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:31.745881 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551e5ff3-30f3-4d95-b76f-f6d8599a75ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:31.746257 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551e5ff3-30f3-4d95-b76f-f6d8599a75ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:31.748276 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551e5ff3-30f3-4d95-b76f-f6d8599a75ad None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:31.760486 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 316/615] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:54:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:31.766750 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdcb168-917c-4f11-b817-532cd1335f20 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:31.771839 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:31.878921 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdcb168-917c-4f11-b817-532cd1335f20 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:31.914557 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13c144d-2c5d-4f24-bbf1-84ec0d7eb4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:31.915225 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13c144d-2c5d-4f24-bbf1-84ec0d7eb4db None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:31.915546 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13c144d-2c5d-4f24-bbf1-84ec0d7eb4db None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:31.915829 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13c144d-2c5d-4f24-bbf1-84ec0d7eb4db None admin] PATH_INFO: `/v3/users/swift` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:31.924432 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13c144d-2c5d-4f24-bbf1-84ec0d7eb4db None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:31.925955 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13c144d-2c5d-4f24-bbf1-84ec0d7eb4db None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:31.937917 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-b13c144d-2c5d-4f24-bbf1-84ec0d7eb4db None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 18 16:54:31.937917 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:31.937917 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:31.937917 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:31.937917 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:31.937917 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:31.937917 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:31.937917 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:31.937917 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:31.937917 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:31.937917 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:31.937917 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:31.937917 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 16:54:31.937917 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 16:54:31.937917 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 16:54:31.937917 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 16:54:31.938840 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:31.938840 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:31.938840 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 16:54:31.938840 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:54:31.938840 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 16:54:31.938840 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:54:31.938840 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:31.938840 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:31.938840 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:31.938840 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:31.938840 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:31.938840 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:31.938840 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:31.938840 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:31.938840 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:31.938840 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:31.938840 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:31.938840 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:31.939816 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:31.939816 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:31.939816 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 16:54:31.939816 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 16:54:31.939816 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 16:54:31.939816 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 16:54:31.939816 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 16:54:31.939816 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 16:54:31.939816 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 18 16:54:31.939816 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:31.939816 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 300/616] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 16:54:31 2022] GET /identity/v3/users/swift => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:31.945221 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551e5ff3-30f3-4d95-b76f-f6d8599a75ad None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:31.963729 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13c144d-2c5d-4f24-bbf1-84ec0d7eb4db None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:32.046321 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:32.100042 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13c144d-2c5d-4f24-bbf1-84ec0d7eb4db None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:32.141269 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551e5ff3-30f3-4d95-b76f-f6d8599a75ad None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:32.166254 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105bf7c9-85af-4fa7-a57a-7bd58ccff3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:32.166887 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105bf7c9-85af-4fa7-a57a-7bd58ccff3d5 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:32.167186 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105bf7c9-85af-4fa7-a57a-7bd58ccff3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:32.167459 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105bf7c9-85af-4fa7-a57a-7bd58ccff3d5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:32.187885 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105bf7c9-85af-4fa7-a57a-7bd58ccff3d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:54:32.192998 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-105bf7c9-85af-4fa7-a57a-7bd58ccff3d5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 16:54:32.192998 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:32.192998 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:32.192998 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:32.192998 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:32.192998 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:32.192998 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:32.192998 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:32.192998 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:32.192998 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:32.192998 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:32.192998 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:32.192998 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:54:32.192998 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:54:32.192998 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:54:32.192998 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:54:32.194064 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:54:32.194064 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:54:32.194064 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:54:32.194064 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:54:32.194064 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:32.194064 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:32.194064 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:32.194064 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:32.194064 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:32.194064 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:32.194064 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:32.194064 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:32.194064 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:32.194064 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:32.194064 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:32.194064 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:32.194963 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:32.194963 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:32.194963 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:32.194963 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:32.194963 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:54:32.194963 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:54:32.194963 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:54:32.194963 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:54:32.194963 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:54:32.194963 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:54:32.194963 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 16:54:32.194963 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:32.202461 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 301/617] 10.222.0.82 () {56 vars in 1102 bytes} [Thu Aug 18 16:54:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 16:54:32.207752 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105bf7c9-85af-4fa7-a57a-7bd58ccff3d5 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:32.213975 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408a052f-4cd8-4a50-a608-667ac6c3905c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:32.214646 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a052f-4cd8-4a50-a608-667ac6c3905c None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:32.214949 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a052f-4cd8-4a50-a608-667ac6c3905c None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:32.215479 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a052f-4cd8-4a50-a608-667ac6c3905c None admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:32.216772 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408a052f-4cd8-4a50-a608-667ac6c3905c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:32.217074 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408a052f-4cd8-4a50-a608-667ac6c3905c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:32.218794 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408a052f-4cd8-4a50-a608-667ac6c3905c None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:32.252381 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 317/618] 10.222.0.82 () {56 vars in 1135 bytes} [Thu Aug 18 16:54:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:32.257436 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408a052f-4cd8-4a50-a608-667ac6c3905c None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:32.272293 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:32.341614 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105bf7c9-85af-4fa7-a57a-7bd58ccff3d5 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:32.381593 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408a052f-4cd8-4a50-a608-667ac6c3905c None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:32.384371 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:32.396544 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23451c2d-abed-4922-b052-6c21269d8e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:32.397150 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23451c2d-abed-4922-b052-6c21269d8e65 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:32.397428 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23451c2d-abed-4922-b052-6c21269d8e65 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:32.397699 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23451c2d-abed-4922-b052-6c21269d8e65 None admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:32.398461 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23451c2d-abed-4922-b052-6c21269d8e65 None admin] RBAC: Adding query filter params (name=service) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:32.398767 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23451c2d-abed-4922-b052-6c21269d8e65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:32.404775 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23451c2d-abed-4922-b052-6c21269d8e65 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:32.422066 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a6d161-818b-4942-a392-38c5ec1aad9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:32.422698 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a6d161-818b-4942-a392-38c5ec1aad9d None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:32.423050 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a6d161-818b-4942-a392-38c5ec1aad9d None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:32.423212 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 302/619] 10.222.0.82 () {56 vars in 1103 bytes} [Thu Aug 18 16:54:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:54:32.423371 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a6d161-818b-4942-a392-38c5ec1aad9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:32.429039 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23451c2d-abed-4922-b052-6c21269d8e65 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:32.436110 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a6d161-818b-4942-a392-38c5ec1aad9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:32.437683 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a6d161-818b-4942-a392-38c5ec1aad9d None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:32.455223 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-94a6d161-818b-4942-a392-38c5ec1aad9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:32.455223 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:32.455223 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:32.455223 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:32.455223 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:32.455223 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:32.455223 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:32.455223 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:32.455223 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:32.455223 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:32.455223 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:32.455223 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:32.455223 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:54:32.455223 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:54:32.455223 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:54:32.455223 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:54:32.456197 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:32.456197 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:32.456197 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:32.456197 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:32.456197 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:32.456197 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:32.456197 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:32.456197 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:32.456197 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:32.456197 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:32.456197 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:32.456197 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:32.456197 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:32.456197 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:32.456197 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:32.456197 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:32.456197 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:54:32.457459 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:54:32.457459 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:32.457459 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:32.457459 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 318/620] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:54:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:32.462402 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a6d161-818b-4942-a392-38c5ec1aad9d None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:32.535719 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:32.610304 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a6d161-818b-4942-a392-38c5ec1aad9d None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:32.658401 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23451c2d-abed-4922-b052-6c21269d8e65 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:32.683069 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026b4fb3-5a80-4f1f-88da-dcaef2270c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:32.683703 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026b4fb3-5a80-4f1f-88da-dcaef2270c06 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:32.683987 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026b4fb3-5a80-4f1f-88da-dcaef2270c06 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:32.684262 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026b4fb3-5a80-4f1f-88da-dcaef2270c06 None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:32.685542 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026b4fb3-5a80-4f1f-88da-dcaef2270c06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:32.685762 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026b4fb3-5a80-4f1f-88da-dcaef2270c06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:32.687213 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026b4fb3-5a80-4f1f-88da-dcaef2270c06 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:32.721828 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 319/621] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:54:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:54:32.722741 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026b4fb3-5a80-4f1f-88da-dcaef2270c06 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:32.741859 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06819232-74be-4e5b-8ed5-215f059c3eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:32.742671 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06819232-74be-4e5b-8ed5-215f059c3eaf None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:32.742997 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06819232-74be-4e5b-8ed5-215f059c3eaf None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:32.743281 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06819232-74be-4e5b-8ed5-215f059c3eaf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:32.757671 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06819232-74be-4e5b-8ed5-215f059c3eaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:32.759443 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06819232-74be-4e5b-8ed5-215f059c3eaf None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:32.776220 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-06819232-74be-4e5b-8ed5-215f059c3eaf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:32.776220 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:32.776220 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:32.776220 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:32.776220 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:32.776220 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:32.776220 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:32.776220 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:32.776220 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:32.776220 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:32.776220 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:32.776220 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:32.776220 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:54:32.776220 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:54:32.776220 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:54:32.776220 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:54:32.777127 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:32.777127 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:32.777127 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:32.777127 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:32.777127 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:32.777127 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:32.777127 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:32.777127 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:32.777127 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:32.777127 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:32.777127 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:32.777127 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:32.777127 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:32.777127 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:32.777127 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:32.777127 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:32.777127 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:54:32.778600 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:54:32.778600 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:32.778600 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:32.778600 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 303/622] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:54:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 16:54:32.785804 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06819232-74be-4e5b-8ed5-215f059c3eaf None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:32.855688 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:32.896507 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06819232-74be-4e5b-8ed5-215f059c3eaf None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:32.932183 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d5c831-2d90-412f-8400-008aad53765c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:32.932807 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d5c831-2d90-412f-8400-008aad53765c None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:32.933098 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d5c831-2d90-412f-8400-008aad53765c None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:32.933381 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d5c831-2d90-412f-8400-008aad53765c None admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:32.934027 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d5c831-2d90-412f-8400-008aad53765c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:32.934340 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d5c831-2d90-412f-8400-008aad53765c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:32.936865 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d5c831-2d90-412f-8400-008aad53765c None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:32.949825 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 304/623] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:54:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:32.956678 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d5c831-2d90-412f-8400-008aad53765c None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:33.014835 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026b4fb3-5a80-4f1f-88da-dcaef2270c06 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:33.043361 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:33.061531 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d2bdb-1751-451e-b948-3d4cdfe9a82e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:33.063306 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d2bdb-1751-451e-b948-3d4cdfe9a82e None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:33.063306 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d2bdb-1751-451e-b948-3d4cdfe9a82e None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:33.063306 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d2bdb-1751-451e-b948-3d4cdfe9a82e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:33.078577 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88d2bdb-1751-451e-b948-3d4cdfe9a82e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:33.082306 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88d2bdb-1751-451e-b948-3d4cdfe9a82e None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:33.114094 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-e88d2bdb-1751-451e-b948-3d4cdfe9a82e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:54:33.114094 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:33.114094 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:33.114094 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:33.114094 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:33.114094 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:33.114094 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:33.114094 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:33.114094 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:33.114094 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:33.114094 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:33.114094 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:33.114094 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 16:54:33.114094 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 16:54:33.114094 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 16:54:33.115280 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 16:54:33.115280 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:33.115280 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:33.115280 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:33.115280 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:33.115280 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:33.115280 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:33.115280 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:33.115280 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:33.115280 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:33.115280 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:33.115280 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:33.115280 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:33.115280 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:33.115280 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:33.115280 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:33.115280 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:33.116379 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 16:54:33.116379 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 16:54:33.116379 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 16:54:33.116379 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 16:54:33.116379 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 16:54:33.116379 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 16:54:33.116379 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:54:33.116379 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:33.118434 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 320/624] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 16:54:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 396 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:33.135468 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d2bdb-1751-451e-b948-3d4cdfe9a82e None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:33.177870 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d5c831-2d90-412f-8400-008aad53765c None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:33.183094 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:33.252970 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeea099-7808-4add-bfdc-f6b214351cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:33.253620 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeea099-7808-4add-bfdc-f6b214351cf5 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:33.253907 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeea099-7808-4add-bfdc-f6b214351cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:33.254183 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeea099-7808-4add-bfdc-f6b214351cf5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:33.271538 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeea099-7808-4add-bfdc-f6b214351cf5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:33.273018 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeea099-7808-4add-bfdc-f6b214351cf5 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:33.307999 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-dbeea099-7808-4add-bfdc-f6b214351cf5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 18 16:54:33.307999 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:33.307999 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:33.307999 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:33.307999 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:33.307999 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:33.307999 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:33.307999 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:33.307999 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:33.307999 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:33.307999 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:33.307999 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:33.307999 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 16:54:33.307999 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 16:54:33.307999 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 16:54:33.307999 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 16:54:33.308992 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:33.308992 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:33.308992 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 16:54:33.308992 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:54:33.308992 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 16:54:33.308992 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:54:33.308992 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:33.308992 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:33.308992 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:33.308992 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:33.308992 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:33.308992 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:33.308992 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:33.308992 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:33.308992 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:33.308992 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:33.308992 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:33.308992 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:33.310595 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:33.310595 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:33.310595 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 16:54:33.310595 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 16:54:33.310595 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 16:54:33.310595 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 16:54:33.310595 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 16:54:33.310595 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 16:54:33.310595 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 18 16:54:33.310595 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:33.310595 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 305/625] 10.222.0.82 () {56 vars in 1093 bytes} [Thu Aug 18 16:54:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:33.311569 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d2bdb-1751-451e-b948-3d4cdfe9a82e None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:33.319878 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeea099-7808-4add-bfdc-f6b214351cf5 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:33.359471 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102dc0e8-7890-4a7e-9e97-89d0b32b265d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:33.360090 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102dc0e8-7890-4a7e-9e97-89d0b32b265d None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:33.360369 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102dc0e8-7890-4a7e-9e97-89d0b32b265d None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:33.360664 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102dc0e8-7890-4a7e-9e97-89d0b32b265d None admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:33.361363 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102dc0e8-7890-4a7e-9e97-89d0b32b265d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:33.361674 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102dc0e8-7890-4a7e-9e97-89d0b32b265d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:33.363127 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102dc0e8-7890-4a7e-9e97-89d0b32b265d None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:33.392003 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 321/626] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 16:54:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:54:33.398266 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102dc0e8-7890-4a7e-9e97-89d0b32b265d None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:33.495346 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:33.512206 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeea099-7808-4add-bfdc-f6b214351cf5 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:33.548684 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd3ea63-5483-450e-82c3-91d9492e9afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:33.549442 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3ea63-5483-450e-82c3-91d9492e9afa None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:33.549825 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3ea63-5483-450e-82c3-91d9492e9afa None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:33.550345 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3ea63-5483-450e-82c3-91d9492e9afa None admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:33.551219 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd3ea63-5483-450e-82c3-91d9492e9afa None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:33.551625 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd3ea63-5483-450e-82c3-91d9492e9afa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:33.553225 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd3ea63-5483-450e-82c3-91d9492e9afa None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:33.561736 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102dc0e8-7890-4a7e-9e97-89d0b32b265d None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:33.577712 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 306/627] 10.222.0.82 () {56 vars in 1133 bytes} [Thu Aug 18 16:54:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:33.579215 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22090db5-9db1-48ab-9260-878640ee5e95 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:33.579604 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22090db5-9db1-48ab-9260-878640ee5e95 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:33.579980 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22090db5-9db1-48ab-9260-878640ee5e95 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:33.582068 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 307/628] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:54:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:54:33.583082 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22090db5-9db1-48ab-9260-878640ee5e95 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:33.601935 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa46b74f-193a-4f21-a011-66436f648068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:33.602164 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:33.602600 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46b74f-193a-4f21-a011-66436f648068 None admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:33.603008 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46b74f-193a-4f21-a011-66436f648068 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:33.603188 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46b74f-193a-4f21-a011-66436f648068 None admin] PATH_INFO: `/v3/projects/2293c0aab61f44849d0d48b214cde7ea/users/1381673f81584f4b8084823e3aa99e2a/roles/5e0f402e1af84b44a723ee0730315d74` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:33.618732 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa46b74f-193a-4f21-a011-66436f648068 None admin] RBAC: Authorizing `identity:create_grant(project_id=2293c0aab61f44849d0d48b214cde7ea, user_id=1381673f81584f4b8084823e3aa99e2a, role_id=5e0f402e1af84b44a723ee0730315d74)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:33.620226 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa46b74f-193a-4f21-a011-66436f648068 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:33.687117 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22090db5-9db1-48ab-9260-878640ee5e95 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:33.688170 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 322/629] 10.222.0.82 () {58 vars in 1439 bytes} [Thu Aug 18 16:54:33 2022] PUT /identity/v3/projects/2293c0aab61f44849d0d48b214cde7ea/users/1381673f81584f4b8084823e3aa99e2a/roles/5e0f402e1af84b44a723ee0730315d74 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 16:54:33.690512 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb6db82-018b-46f3-8013-efef4808a616 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:33.690804 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb6db82-018b-46f3-8013-efef4808a616 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:33.691106 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb6db82-018b-46f3-8013-efef4808a616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:33.715665 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-0fb6db82-018b-46f3-8013-efef4808a616 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:54:33.717857 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-0fb6db82-018b-46f3-8013-efef4808a616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:54:33.724629 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ecc1ca-a4f7-440f-aec4-cf2cf27c299a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:33.725291 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ecc1ca-a4f7-440f-aec4-cf2cf27c299a None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:33.725541 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ecc1ca-a4f7-440f-aec4-cf2cf27c299a None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:33.725808 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ecc1ca-a4f7-440f-aec4-cf2cf27c299a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:33.808601 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ecc1ca-a4f7-440f-aec4-cf2cf27c299a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:33.810946 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ecc1ca-a4f7-440f-aec4-cf2cf27c299a None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:33.819074 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 323/630] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:54:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:54:33.819074 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0591f033-501e-468b-b201-9933a810aa20 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:33.819074 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0591f033-501e-468b-b201-9933a810aa20 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:33.819074 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0591f033-501e-468b-b201-9933a810aa20 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:33.819074 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 324/631] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:54:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:54:33.819074 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d708d-2f4c-4397-9da8-038636018a9c None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:33.819074 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d708d-2f4c-4397-9da8-038636018a9c None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:33.819730 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d708d-2f4c-4397-9da8-038636018a9c None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:33.819730 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 325/632] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:54:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:54:33.822931 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a7b0ca-c725-4966-bff8-a57805f695af None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:33.822931 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a7b0ca-c725-4966-bff8-a57805f695af None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:33.822931 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a7b0ca-c725-4966-bff8-a57805f695af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:33.843439 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-a9a7b0ca-c725-4966-bff8-a57805f695af None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:54:33.843630 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:33.844646 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-a9a7b0ca-c725-4966-bff8-a57805f695af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:54:33.864254 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:33.936101 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 326/633] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:54:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:54:33.937389 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba3fa6-f109-4b11-95ce-47492453a44c None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:33.937733 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba3fa6-f109-4b11-95ce-47492453a44c None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:33.938018 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba3fa6-f109-4b11-95ce-47492453a44c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:33.947744 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 308/634] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:54:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 16:54:33.954400 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ecc1ca-a4f7-440f-aec4-cf2cf27c299a None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:33.957852 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-4aba3fa6-f109-4b11-95ce-47492453a44c None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:54:33.959076 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-4aba3fa6-f109-4b11-95ce-47492453a44c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:54:34.036767 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 327/635] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:54:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:54:34.111543 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ecc1ca-a4f7-440f-aec4-cf2cf27c299a None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:34.126408 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aba3fa6-f109-4b11-95ce-47492453a44c None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:34.129895 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:34.157172 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead723a1-42f5-4763-8d42-4cf132442868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:34.157830 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead723a1-42f5-4763-8d42-4cf132442868 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:34.158112 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead723a1-42f5-4763-8d42-4cf132442868 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:34.158412 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead723a1-42f5-4763-8d42-4cf132442868 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:34.177782 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead723a1-42f5-4763-8d42-4cf132442868 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:34.179294 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead723a1-42f5-4763-8d42-4cf132442868 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:34.199666 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-ead723a1-42f5-4763-8d42-4cf132442868 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:34.199666 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:34.199666 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:34.199666 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:34.199666 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:34.199666 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:34.199666 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:34.199666 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:34.199666 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:34.199666 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:34.199666 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:34.199666 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:34.199666 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:54:34.199666 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:54:34.199666 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:54:34.199666 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:54:34.200655 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:34.200655 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:34.200655 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:34.200655 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:34.200655 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:34.200655 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:34.200655 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:34.200655 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:34.200655 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:34.200655 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:34.200655 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:34.200655 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:34.200655 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:34.200655 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:34.200655 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:34.200655 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:34.200655 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:54:34.202119 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:54:34.202119 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:34.202119 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:34.202119 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 309/636] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:54:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 16:54:34.206388 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead723a1-42f5-4763-8d42-4cf132442868 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:34.253937 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:34.274045 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aba3fa6-f109-4b11-95ce-47492453a44c None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:34.312367 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3cfde4-0e64-439c-a671-1080faad75cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:34.313409 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3cfde4-0e64-439c-a671-1080faad75cd None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:34.313885 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3cfde4-0e64-439c-a671-1080faad75cd None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:34.314370 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3cfde4-0e64-439c-a671-1080faad75cd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:34.339777 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3cfde4-0e64-439c-a671-1080faad75cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:54:34.349495 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-fb3cfde4-0e64-439c-a671-1080faad75cd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 16:54:34.349495 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:34.349495 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:34.349495 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:34.349495 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:34.349495 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:34.349495 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:34.349495 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:34.349495 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:34.349495 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:34.349495 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:34.349495 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:34.349495 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:54:34.349495 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:54:34.349495 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:54:34.349495 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:54:34.350496 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:54:34.350496 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:54:34.350496 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:54:34.350496 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:54:34.350496 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:34.350496 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:34.350496 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:34.350496 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:34.350496 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:34.350496 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:34.350496 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:34.350496 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:34.350496 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:34.350496 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:34.350496 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:34.350496 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:34.351436 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:34.351436 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:34.351436 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:34.351436 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:34.351436 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:54:34.351436 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:54:34.351436 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:54:34.351436 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:54:34.351436 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:54:34.351436 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:54:34.351436 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 16:54:34.351436 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:34.351436 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 328/637] 10.222.0.82 () {56 vars in 1102 bytes} [Thu Aug 18 16:54:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 16:54:34.352126 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3cfde4-0e64-439c-a671-1080faad75cd None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:34.366824 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead723a1-42f5-4763-8d42-4cf132442868 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:34.402748 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3137907f-0c9c-4a4a-b69c-aeb31fcc93bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:34.403435 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3137907f-0c9c-4a4a-b69c-aeb31fcc93bb None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:34.403716 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3137907f-0c9c-4a4a-b69c-aeb31fcc93bb None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:34.404082 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3137907f-0c9c-4a4a-b69c-aeb31fcc93bb None admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:34.404628 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3137907f-0c9c-4a4a-b69c-aeb31fcc93bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:34.404934 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3137907f-0c9c-4a4a-b69c-aeb31fcc93bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:34.406311 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3137907f-0c9c-4a4a-b69c-aeb31fcc93bb None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:34.424004 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 310/638] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:54:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:54:34.424004 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3137907f-0c9c-4a4a-b69c-aeb31fcc93bb None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:34.450472 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:34.474797 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3cfde4-0e64-439c-a671-1080faad75cd None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:34.512148 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076651b8-aaaa-4b9d-84b7-b46d22a41e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:34.512799 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076651b8-aaaa-4b9d-84b7-b46d22a41e5d None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:34.513131 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076651b8-aaaa-4b9d-84b7-b46d22a41e5d None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:34.513525 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076651b8-aaaa-4b9d-84b7-b46d22a41e5d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:34.524670 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076651b8-aaaa-4b9d-84b7-b46d22a41e5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:54:34.526122 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3137907f-0c9c-4a4a-b69c-aeb31fcc93bb None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:34.529900 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-076651b8-aaaa-4b9d-84b7-b46d22a41e5d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 16:54:34.529900 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:34.529900 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:34.529900 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:34.529900 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:34.529900 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:34.529900 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:34.529900 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:34.529900 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:34.529900 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:34.529900 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:34.529900 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:34.529900 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:54:34.529900 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:54:34.529900 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:54:34.529900 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:54:34.530808 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:54:34.530808 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:54:34.530808 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:54:34.530808 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:54:34.530808 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:34.530808 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:34.530808 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:34.530808 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:34.530808 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:34.530808 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:34.530808 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:34.530808 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:34.530808 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:34.530808 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:34.530808 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:34.530808 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:34.536089 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:34.536089 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:34.536089 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:34.536089 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:34.536089 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:54:34.536089 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:54:34.536089 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:54:34.536089 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:54:34.536089 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:54:34.536089 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:54:34.536089 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 16:54:34.536089 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:34.536089 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 329/639] 10.222.0.82 () {56 vars in 1102 bytes} [Thu Aug 18 16:54:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:34.536089 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076651b8-aaaa-4b9d-84b7-b46d22a41e5d None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:34.539469 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:34.565921 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fd34bb-5f2e-4a91-b7d8-378670e62921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:34.566530 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fd34bb-5f2e-4a91-b7d8-378670e62921 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:34.566816 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fd34bb-5f2e-4a91-b7d8-378670e62921 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:34.567182 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fd34bb-5f2e-4a91-b7d8-378670e62921 None admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:34.567841 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fd34bb-5f2e-4a91-b7d8-378670e62921 None admin] RBAC: Adding query filter params (name=service) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:34.568145 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fd34bb-5f2e-4a91-b7d8-378670e62921 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:34.569522 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fd34bb-5f2e-4a91-b7d8-378670e62921 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:34.577734 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 311/640] 10.222.0.82 () {56 vars in 1103 bytes} [Thu Aug 18 16:54:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:34.578526 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fd34bb-5f2e-4a91-b7d8-378670e62921 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:34.643826 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076651b8-aaaa-4b9d-84b7-b46d22a41e5d None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:34.672847 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:34.678696 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7f4be7-536a-428e-970d-d297d89003ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:34.679372 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f4be7-536a-428e-970d-d297d89003ab None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:34.679624 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f4be7-536a-428e-970d-d297d89003ab None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:34.679982 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f4be7-536a-428e-970d-d297d89003ab None admin] PATH_INFO: `/v3/roles/service` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:34.683126 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fd34bb-5f2e-4a91-b7d8-378670e62921 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:34.690366 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7f4be7-536a-428e-970d-d297d89003ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:54:34.695474 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-3c7f4be7-536a-428e-970d-d297d89003ab None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 16:54:34.695474 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:34.695474 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:34.695474 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:34.695474 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:34.695474 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:34.695474 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:34.695474 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:34.695474 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:34.695474 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:34.695474 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:34.695474 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:34.695474 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:54:34.695474 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:54:34.695474 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:54:34.695474 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:54:34.696576 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:54:34.696576 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:54:34.696576 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:54:34.696576 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:54:34.696576 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:34.696576 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:34.696576 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:34.696576 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:34.696576 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:34.696576 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:34.696576 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:34.696576 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:34.696576 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:34.696576 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:34.696576 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:34.696576 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:34.700041 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:34.700041 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:34.700041 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:34.700041 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:34.700041 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:54:34.700041 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:54:34.700041 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:54:34.700041 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:54:34.700041 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:54:34.700041 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:54:34.700041 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 16:54:34.700041 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:34.700041 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 330/641] 10.222.0.82 () {56 vars in 1102 bytes} [Thu Aug 18 16:54:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:34.700041 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7f4be7-536a-428e-970d-d297d89003ab None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:34.729013 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e9c03e-83c4-457e-8098-014fda85339f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:34.729645 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e9c03e-83c4-457e-8098-014fda85339f None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:34.729927 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e9c03e-83c4-457e-8098-014fda85339f None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:34.730197 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e9c03e-83c4-457e-8098-014fda85339f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:34.737574 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e9c03e-83c4-457e-8098-014fda85339f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:34.739114 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e9c03e-83c4-457e-8098-014fda85339f None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:34.750322 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-d3e9c03e-83c4-457e-8098-014fda85339f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:54:34.750322 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:34.750322 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:34.750322 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:34.750322 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:34.750322 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:34.750322 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:34.750322 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:34.750322 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:34.750322 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:34.750322 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:34.750322 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:34.750322 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 16:54:34.750322 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 16:54:34.750322 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 16:54:34.751479 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 16:54:34.751479 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:34.751479 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:34.751479 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:34.751479 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:34.751479 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:34.751479 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:34.751479 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:34.751479 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:34.751479 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:34.751479 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:34.751479 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:34.751479 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:34.751479 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:34.751479 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:34.751479 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:34.751479 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:34.752387 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 16:54:34.752387 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 16:54:34.752387 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 16:54:34.752387 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 16:54:34.752387 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 16:54:34.752387 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 16:54:34.752387 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:54:34.752387 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:34.752387 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 312/642] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 16:54:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:34.752387 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e9c03e-83c4-457e-8098-014fda85339f None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:34.770981 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:34.817928 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7f4be7-536a-428e-970d-d297d89003ab None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:34.856033 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260d69ca-7ea4-4698-88c4-9f7a4b6c3536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:34.856900 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d69ca-7ea4-4698-88c4-9f7a4b6c3536 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:34.857157 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d69ca-7ea4-4698-88c4-9f7a4b6c3536 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:34.857434 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d69ca-7ea4-4698-88c4-9f7a4b6c3536 None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:34.858090 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260d69ca-7ea4-4698-88c4-9f7a4b6c3536 None admin] RBAC: Adding query filter params (name=service) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:34.858389 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260d69ca-7ea4-4698-88c4-9f7a4b6c3536 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:34.859830 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260d69ca-7ea4-4698-88c4-9f7a4b6c3536 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:34.868326 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 331/643] 10.222.0.82 () {56 vars in 1103 bytes} [Thu Aug 18 16:54:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:34.869072 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260d69ca-7ea4-4698-88c4-9f7a4b6c3536 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:34.878539 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e9c03e-83c4-457e-8098-014fda85339f None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:34.915436 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a924d0d1-dff2-4e93-9d77-7ab6bdfbd183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:34.916109 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a924d0d1-dff2-4e93-9d77-7ab6bdfbd183 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:34.916398 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a924d0d1-dff2-4e93-9d77-7ab6bdfbd183 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:34.916678 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a924d0d1-dff2-4e93-9d77-7ab6bdfbd183 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:34.928222 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a924d0d1-dff2-4e93-9d77-7ab6bdfbd183 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:34.929682 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a924d0d1-dff2-4e93-9d77-7ab6bdfbd183 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:34.944624 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-a924d0d1-dff2-4e93-9d77-7ab6bdfbd183 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:34.944624 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:34.944624 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:34.944624 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:34.944624 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:34.944624 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:34.944624 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:34.944624 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:34.944624 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:34.944624 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:34.944624 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:34.944624 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:34.944624 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:54:34.944624 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:54:34.944624 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:54:34.944624 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:54:34.945538 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:34.945538 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:34.945538 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:34.945538 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:34.945538 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:34.945538 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:34.945538 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:34.945538 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:34.945538 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:34.945538 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:34.945538 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:34.945538 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:34.945538 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:34.945538 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:34.945538 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:34.945538 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:34.945538 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:54:34.946560 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:54:34.946560 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:34.946560 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:34.946560 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 313/644] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:54:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:34.946560 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a924d0d1-dff2-4e93-9d77-7ab6bdfbd183 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:34.949467 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:34.988464 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260d69ca-7ea4-4698-88c4-9f7a4b6c3536 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:35.014609 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:35.025262 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0098c66b-8b89-44f4-af78-29c44ddf4733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:35.025866 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0098c66b-8b89-44f4-af78-29c44ddf4733 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:35.026142 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0098c66b-8b89-44f4-af78-29c44ddf4733 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:35.026414 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0098c66b-8b89-44f4-af78-29c44ddf4733 None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:35.027091 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0098c66b-8b89-44f4-af78-29c44ddf4733 None admin] RBAC: Adding query filter params (name=service) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:35.027393 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0098c66b-8b89-44f4-af78-29c44ddf4733 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:35.028827 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0098c66b-8b89-44f4-af78-29c44ddf4733 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:35.037007 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 332/645] 10.222.0.82 () {56 vars in 1103 bytes} [Thu Aug 18 16:54:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:35.037789 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0098c66b-8b89-44f4-af78-29c44ddf4733 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:35.055550 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a924d0d1-dff2-4e93-9d77-7ab6bdfbd183 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:35.091887 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eba6e91-70ad-4c05-9e8a-bc618a47b837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:35.092411 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba6e91-70ad-4c05-9e8a-bc618a47b837 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:35.092697 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba6e91-70ad-4c05-9e8a-bc618a47b837 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:35.092970 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba6e91-70ad-4c05-9e8a-bc618a47b837 None admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:35.093665 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eba6e91-70ad-4c05-9e8a-bc618a47b837 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:35.093971 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eba6e91-70ad-4c05-9e8a-bc618a47b837 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:35.095447 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eba6e91-70ad-4c05-9e8a-bc618a47b837 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:35.112728 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 314/646] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 16:54:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:35.112728 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eba6e91-70ad-4c05-9e8a-bc618a47b837 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:35.145991 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0098c66b-8b89-44f4-af78-29c44ddf4733 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:35.173996 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:35.181140 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6f431b-d03d-49c5-b65c-fb22c12f1fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:35.181951 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6f431b-d03d-49c5-b65c-fb22c12f1fc0 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:35.182415 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6f431b-d03d-49c5-b65c-fb22c12f1fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:35.182743 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6f431b-d03d-49c5-b65c-fb22c12f1fc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:35.194568 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6f431b-d03d-49c5-b65c-fb22c12f1fc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:35.196101 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6f431b-d03d-49c5-b65c-fb22c12f1fc0 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:35.211727 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-1d6f431b-d03d-49c5-b65c-fb22c12f1fc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:35.211727 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:35.211727 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:35.211727 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:35.211727 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:35.211727 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:35.211727 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:35.211727 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:35.211727 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:35.211727 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:35.211727 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:35.211727 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:35.211727 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:54:35.211727 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:54:35.211727 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:54:35.211727 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:54:35.212725 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:35.212725 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:35.212725 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:35.212725 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:35.212725 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:35.212725 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:35.212725 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:35.212725 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:35.212725 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:35.212725 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:35.212725 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:35.212725 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:35.212725 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:35.212725 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:35.212725 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:35.212725 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:35.212725 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:54:35.214066 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:54:35.214066 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:35.214066 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:35.214066 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 333/647] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:54:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:35.214283 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6f431b-d03d-49c5-b65c-fb22c12f1fc0 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:35.223657 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eba6e91-70ad-4c05-9e8a-bc618a47b837 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:35.258295 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:35.261102 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39390eb5-504c-48a6-babf-574ae19f29e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:35.261734 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39390eb5-504c-48a6-babf-574ae19f29e1 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:35.262023 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39390eb5-504c-48a6-babf-574ae19f29e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:35.262302 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39390eb5-504c-48a6-babf-574ae19f29e1 None admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:35.263223 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39390eb5-504c-48a6-babf-574ae19f29e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:35.263727 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39390eb5-504c-48a6-babf-574ae19f29e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:35.265121 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39390eb5-504c-48a6-babf-574ae19f29e1 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:35.276841 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 315/648] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:54:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:35.277664 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39390eb5-504c-48a6-babf-574ae19f29e1 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:35.326410 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6f431b-d03d-49c5-b65c-fb22c12f1fc0 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:35.364393 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705140d1-1dbf-4015-b92f-ce1735fe4045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:35.364947 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705140d1-1dbf-4015-b92f-ce1735fe4045 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:35.365230 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705140d1-1dbf-4015-b92f-ce1735fe4045 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:35.365510 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705140d1-1dbf-4015-b92f-ce1735fe4045 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:35.377659 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705140d1-1dbf-4015-b92f-ce1735fe4045 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:35.379162 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705140d1-1dbf-4015-b92f-ce1735fe4045 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:35.385288 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39390eb5-504c-48a6-babf-574ae19f29e1 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:35.393707 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-705140d1-1dbf-4015-b92f-ce1735fe4045 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:35.393707 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:35.393707 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:35.393707 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:35.393707 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:35.393707 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:35.393707 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:35.393707 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:35.393707 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:35.393707 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:35.393707 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:35.393707 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:35.393707 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:54:35.393707 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:54:35.393707 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:54:35.393707 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:54:35.394626 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:35.394626 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:35.394626 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:35.394626 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:35.394626 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:35.394626 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:35.394626 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:35.394626 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:35.394626 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:35.394626 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:35.394626 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:35.394626 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:35.394626 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:35.394626 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:35.394626 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:35.394626 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:35.394626 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:54:35.395513 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:54:35.395513 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:35.395513 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:35.395513 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 334/649] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:54:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:35.396213 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705140d1-1dbf-4015-b92f-ce1735fe4045 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:35.415799 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:35.428763 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d998e46-4d24-4e08-823e-8150a3d701fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:35.429460 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d998e46-4d24-4e08-823e-8150a3d701fc None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:35.429750 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d998e46-4d24-4e08-823e-8150a3d701fc None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:35.430030 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d998e46-4d24-4e08-823e-8150a3d701fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:35.430756 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d998e46-4d24-4e08-823e-8150a3d701fc None admin] RBAC: Adding query filter params (role.id=5e0f402e1af84b44a723ee0730315d74, scope.project.id=2293c0aab61f44849d0d48b214cde7ea, user.id=55723c756cae4e7ab636054db48de02d) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:35.431101 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d998e46-4d24-4e08-823e-8150a3d701fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:35.432681 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d998e46-4d24-4e08-823e-8150a3d701fc None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:35.438989 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 316/650] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 16:54:35 2022] GET /identity/v3/role_assignments?user.id=55723c756cae4e7ab636054db48de02d&scope.project.id=2293c0aab61f44849d0d48b214cde7ea&role.id=5e0f402e1af84b44a723ee0730315d74 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:35.439794 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d998e46-4d24-4e08-823e-8150a3d701fc None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:35.516543 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:35.529288 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705140d1-1dbf-4015-b92f-ce1735fe4045 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:35.566119 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d998e46-4d24-4e08-823e-8150a3d701fc None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:35.581434 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58429a0-458a-49c2-84e2-91c573e0c8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:35.582061 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58429a0-458a-49c2-84e2-91c573e0c8dd None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:35.582340 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58429a0-458a-49c2-84e2-91c573e0c8dd None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:35.582629 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58429a0-458a-49c2-84e2-91c573e0c8dd None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:35.583420 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58429a0-458a-49c2-84e2-91c573e0c8dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:35.583733 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58429a0-458a-49c2-84e2-91c573e0c8dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:35.585245 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58429a0-458a-49c2-84e2-91c573e0c8dd None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:35.598932 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 335/651] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:54:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:35.599916 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58429a0-458a-49c2-84e2-91c573e0c8dd None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:35.608673 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f41f2e-3572-45d3-915c-1456b3a3786f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:35.609328 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f41f2e-3572-45d3-915c-1456b3a3786f None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:35.609651 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f41f2e-3572-45d3-915c-1456b3a3786f None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:35.610008 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f41f2e-3572-45d3-915c-1456b3a3786f None admin] PATH_INFO: `/v3/users/glance` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:35.619177 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f41f2e-3572-45d3-915c-1456b3a3786f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:35.620997 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f41f2e-3572-45d3-915c-1456b3a3786f None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:35.636761 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-a8f41f2e-3572-45d3-915c-1456b3a3786f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 18 16:54:35.636761 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:35.636761 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:35.636761 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:35.636761 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:35.636761 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:35.636761 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:35.636761 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:35.636761 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:35.636761 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:35.636761 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:35.636761 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:35.636761 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 16:54:35.636761 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 16:54:35.636761 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 16:54:35.636761 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 16:54:35.638296 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:35.638296 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:35.638296 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 16:54:35.638296 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:54:35.638296 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 16:54:35.638296 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:54:35.638296 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:35.638296 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:35.638296 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:35.638296 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:35.638296 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:35.638296 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:35.638296 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:35.638296 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:35.638296 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:35.638296 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:35.638296 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:35.638296 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:35.639341 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:35.639341 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:35.639341 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 16:54:35.639341 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 16:54:35.639341 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 16:54:35.639341 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 16:54:35.639341 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 16:54:35.639341 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 16:54:35.639341 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 18 16:54:35.639341 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:35.640380 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 317/652] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:54:35 2022] GET /identity/v3/users/glance => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:35.641124 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f41f2e-3572-45d3-915c-1456b3a3786f None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:35.703802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:35.722153 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58429a0-458a-49c2-84e2-91c573e0c8dd None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:35.759602 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b9a2f0-797f-4da4-9485-7781000f0e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:35.760221 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b9a2f0-797f-4da4-9485-7781000f0e5d None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:35.760506 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b9a2f0-797f-4da4-9485-7781000f0e5d None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:35.760783 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b9a2f0-797f-4da4-9485-7781000f0e5d None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:35.761422 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b9a2f0-797f-4da4-9485-7781000f0e5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:35.761726 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b9a2f0-797f-4da4-9485-7781000f0e5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:35.765952 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b9a2f0-797f-4da4-9485-7781000f0e5d None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:35.773972 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f41f2e-3572-45d3-915c-1456b3a3786f None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:35.777428 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 336/653] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:54:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:35.778501 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b9a2f0-797f-4da4-9485-7781000f0e5d None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:35.792468 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:35.816379 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0205ad09-4413-4943-97d7-55ff95fbc634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:35.817165 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0205ad09-4413-4943-97d7-55ff95fbc634 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:35.817549 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0205ad09-4413-4943-97d7-55ff95fbc634 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:35.817939 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0205ad09-4413-4943-97d7-55ff95fbc634 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:35.826687 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0205ad09-4413-4943-97d7-55ff95fbc634 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:35.828201 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0205ad09-4413-4943-97d7-55ff95fbc634 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:35.839129 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-0205ad09-4413-4943-97d7-55ff95fbc634 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 18 16:54:35.839129 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:35.839129 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:35.839129 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:35.839129 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:35.839129 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:35.839129 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:35.839129 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:35.839129 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:35.839129 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:35.839129 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:35.839129 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:35.839129 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 16:54:35.839129 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 16:54:35.839129 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 16:54:35.839129 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 16:54:35.840494 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:35.840494 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:35.840494 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 16:54:35.840494 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:54:35.840494 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 16:54:35.840494 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:54:35.840494 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:35.840494 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:35.840494 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:35.840494 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:35.840494 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:35.840494 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:35.840494 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:35.840494 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:35.840494 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:35.840494 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:35.840494 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:35.840494 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:35.842383 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:35.842383 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:35.842383 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 16:54:35.842383 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 16:54:35.842383 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 16:54:35.842383 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 16:54:35.842383 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 16:54:35.842383 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 16:54:35.842383 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 18 16:54:35.842383 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:35.842383 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 318/654] 10.222.0.82 () {56 vars in 1102 bytes} [Thu Aug 18 16:54:35 2022] GET /identity/v3/users/neutron => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:35.842991 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d5356c-dd53-4ece-a5dd-1b7013132ba1 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:35.843391 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d5356c-dd53-4ece-a5dd-1b7013132ba1 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:35.843674 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d5356c-dd53-4ece-a5dd-1b7013132ba1 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:35.845059 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 319/655] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:54:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:54:35.847946 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d5356c-dd53-4ece-a5dd-1b7013132ba1 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:35.894274 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b9a2f0-797f-4da4-9485-7781000f0e5d None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:35.932176 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138d0936-dda4-4abc-a29b-15c3380e020b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:35.932962 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d0936-dda4-4abc-a29b-15c3380e020b None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:35.933362 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d0936-dda4-4abc-a29b-15c3380e020b None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:35.933689 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d0936-dda4-4abc-a29b-15c3380e020b None admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:35.934646 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138d0936-dda4-4abc-a29b-15c3380e020b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:35.935126 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138d0936-dda4-4abc-a29b-15c3380e020b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:35.936626 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138d0936-dda4-4abc-a29b-15c3380e020b None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:35.959331 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 337/656] 10.222.0.82 () {56 vars in 1137 bytes} [Thu Aug 18 16:54:35 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:35.959331 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138d0936-dda4-4abc-a29b-15c3380e020b None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:35.960922 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d5356c-dd53-4ece-a5dd-1b7013132ba1 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:35.970022 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:35.998801 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c794439-0a5a-44b5-9760-889fcf6bf02f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:35.999716 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c794439-0a5a-44b5-9760-889fcf6bf02f None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:35.999992 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c794439-0a5a-44b5-9760-889fcf6bf02f None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:36.000288 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c794439-0a5a-44b5-9760-889fcf6bf02f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:36.007644 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c794439-0a5a-44b5-9760-889fcf6bf02f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:36.009123 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c794439-0a5a-44b5-9760-889fcf6bf02f None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:36.024522 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-7c794439-0a5a-44b5-9760-889fcf6bf02f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 18 16:54:36.024522 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:36.024522 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:36.024522 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:36.024522 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:36.024522 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:36.024522 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:36.024522 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:36.024522 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:36.024522 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:36.024522 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:36.024522 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:36.024522 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 16:54:36.024522 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 16:54:36.024522 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 16:54:36.024522 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 16:54:36.025820 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:36.025820 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:36.025820 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 16:54:36.025820 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:54:36.025820 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 16:54:36.025820 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:54:36.025820 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:36.025820 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:36.025820 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:36.025820 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:36.025820 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:36.025820 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:36.025820 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:36.025820 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:36.025820 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:36.025820 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:36.025820 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:36.025820 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:36.026726 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:36.026726 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:36.026726 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 16:54:36.026726 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 16:54:36.026726 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 16:54:36.026726 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 16:54:36.026726 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 16:54:36.026726 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 16:54:36.026726 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 18 16:54:36.026726 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:36.026726 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 320/657] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:54:35 2022] GET /identity/v3/users/cinder => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:36.026726 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c10ddc7-33b8-4212-92a4-20abea97d3c8 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:36.026726 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c10ddc7-33b8-4212-92a4-20abea97d3c8 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:36.027821 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c10ddc7-33b8-4212-92a4-20abea97d3c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:36.035023 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:36.045776 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-7c10ddc7-33b8-4212-92a4-20abea97d3c8 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:54:36.053076 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-7c10ddc7-33b8-4212-92a4-20abea97d3c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:54:36.070561 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138d0936-dda4-4abc-a29b-15c3380e020b None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:36.112923 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf60c0ec-7952-4f6c-96d7-296c7531e143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:36.113630 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf60c0ec-7952-4f6c-96d7-296c7531e143 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:36.113912 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf60c0ec-7952-4f6c-96d7-296c7531e143 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:36.114192 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf60c0ec-7952-4f6c-96d7-296c7531e143 None admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:36.114929 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf60c0ec-7952-4f6c-96d7-296c7531e143 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:36.115240 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf60c0ec-7952-4f6c-96d7-296c7531e143 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:36.116657 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf60c0ec-7952-4f6c-96d7-296c7531e143 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:36.127155 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 321/658] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:54:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:54:36.127155 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c10ddc7-33b8-4212-92a4-20abea97d3c8 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:36.139241 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 338/659] 10.222.0.82 () {56 vars in 1139 bytes} [Thu Aug 18 16:54:35 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:36.140132 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf60c0ec-7952-4f6c-96d7-296c7531e143 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:36.242804 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf60c0ec-7952-4f6c-96d7-296c7531e143 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:36.253779 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:36.260566 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c10ddc7-33b8-4212-92a4-20abea97d3c8 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:36.270672 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:36.280372 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a43f03d-4b07-48fd-82a4-f9cd56f81bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:36.280984 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a43f03d-4b07-48fd-82a4-f9cd56f81bb7 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:36.281313 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a43f03d-4b07-48fd-82a4-f9cd56f81bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:36.281594 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a43f03d-4b07-48fd-82a4-f9cd56f81bb7 None admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:36.282322 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a43f03d-4b07-48fd-82a4-f9cd56f81bb7 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:36.282634 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a43f03d-4b07-48fd-82a4-f9cd56f81bb7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:36.284102 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a43f03d-4b07-48fd-82a4-f9cd56f81bb7 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:36.304763 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 339/660] 10.222.0.82 () {56 vars in 1137 bytes} [Thu Aug 18 16:54:36 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:36.305703 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a43f03d-4b07-48fd-82a4-f9cd56f81bb7 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:36.308736 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f1e841-b6a3-4753-8d1f-5bfe00d1c5f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:36.310983 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1e841-b6a3-4753-8d1f-5bfe00d1c5f6 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:36.311429 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1e841-b6a3-4753-8d1f-5bfe00d1c5f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:36.311841 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1e841-b6a3-4753-8d1f-5bfe00d1c5f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:36.397737 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f1e841-b6a3-4753-8d1f-5bfe00d1c5f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:36.399430 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f1e841-b6a3-4753-8d1f-5bfe00d1c5f6 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:36.409869 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a43f03d-4b07-48fd-82a4-f9cd56f81bb7 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:36.445651 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde3148e-e2bb-414d-8326-87dadad9baae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:36.446256 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde3148e-e2bb-414d-8326-87dadad9baae None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:36.446537 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde3148e-e2bb-414d-8326-87dadad9baae None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:36.446807 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde3148e-e2bb-414d-8326-87dadad9baae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:36.501685 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:36.507413 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:36.513625 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 322/661] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:54:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 16:54:36.513625 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f1e841-b6a3-4753-8d1f-5bfe00d1c5f6 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:36.524324 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde3148e-e2bb-414d-8326-87dadad9baae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:36.525770 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde3148e-e2bb-414d-8326-87dadad9baae None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:36.619644 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f1e841-b6a3-4753-8d1f-5bfe00d1c5f6 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:36.634544 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 340/662] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:54:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 16:54:36.635674 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde3148e-e2bb-414d-8326-87dadad9baae None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:36.663157 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e9dd0e-552c-4904-818c-323512b4529b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:36.663784 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e9dd0e-552c-4904-818c-323512b4529b None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:36.664068 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e9dd0e-552c-4904-818c-323512b4529b None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:36.664347 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e9dd0e-552c-4904-818c-323512b4529b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:36.727571 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:36.730199 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:36.734844 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e9dd0e-552c-4904-818c-323512b4529b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:36.742477 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e9dd0e-552c-4904-818c-323512b4529b None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:36.748542 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde3148e-e2bb-414d-8326-87dadad9baae None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:36.792289 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0cc383-2fc4-4a67-b073-f9e9e2a030f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:36.792918 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0cc383-2fc4-4a67-b073-f9e9e2a030f5 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:36.793202 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0cc383-2fc4-4a67-b073-f9e9e2a030f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:36.793476 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0cc383-2fc4-4a67-b073-f9e9e2a030f5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:36.805085 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0cc383-2fc4-4a67-b073-f9e9e2a030f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:54:36.810950 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-0c0cc383-2fc4-4a67-b073-f9e9e2a030f5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 16:54:36.810950 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:36.810950 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:36.810950 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:36.810950 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:36.810950 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:36.810950 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:36.810950 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:36.810950 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:36.810950 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:36.810950 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:36.810950 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:36.810950 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:54:36.810950 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:54:36.810950 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:54:36.810950 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:54:36.811855 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:54:36.811855 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:54:36.811855 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:54:36.811855 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:54:36.811855 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:36.811855 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:36.811855 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:36.811855 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:36.811855 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:36.811855 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:36.811855 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:36.811855 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:36.811855 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:36.811855 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:36.811855 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:36.811855 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:36.812669 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:36.812669 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:36.812669 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:36.812669 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:36.812669 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:54:36.812669 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:54:36.812669 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:54:36.812669 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:54:36.812669 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:54:36.812669 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:54:36.812669 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 16:54:36.812669 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:36.812669 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 341/663] 10.222.0.82 () {56 vars in 1102 bytes} [Thu Aug 18 16:54:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:36.813430 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0cc383-2fc4-4a67-b073-f9e9e2a030f5 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:36.846981 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 323/664] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:54:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 16:54:36.847825 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e9dd0e-552c-4904-818c-323512b4529b None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:36.925076 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0cc383-2fc4-4a67-b073-f9e9e2a030f5 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:36.949026 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:36.957479 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e9dd0e-552c-4904-818c-323512b4529b None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:36.960900 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b611af21-ca9a-4645-9d0e-e986c446bca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:36.961533 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611af21-ca9a-4645-9d0e-e986c446bca5 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:36.961818 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611af21-ca9a-4645-9d0e-e986c446bca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:36.962096 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611af21-ca9a-4645-9d0e-e986c446bca5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:36.973752 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b611af21-ca9a-4645-9d0e-e986c446bca5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:36.975230 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b611af21-ca9a-4645-9d0e-e986c446bca5 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:36.978044 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:36.990801 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-b611af21-ca9a-4645-9d0e-e986c446bca5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:36.990801 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:36.990801 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:36.990801 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:36.990801 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:36.990801 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:36.990801 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:36.990801 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:36.990801 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:36.990801 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:36.990801 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:36.990801 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:36.990801 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:54:36.990801 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:54:36.990801 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:54:36.990801 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:54:36.991782 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:36.991782 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:36.991782 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:36.991782 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:36.991782 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:36.991782 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:36.991782 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:36.991782 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:36.991782 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:36.991782 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:36.991782 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:36.991782 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:36.991782 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:36.991782 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:36.991782 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:36.991782 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:36.991782 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:54:36.992642 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:54:36.992642 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:36.992642 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:36.992642 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 342/665] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:54:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:36.992870 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b611af21-ca9a-4645-9d0e-e986c446bca5 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:37.002247 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b1bcac-1066-4596-ae8e-4dbbe1d28ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:37.002889 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b1bcac-1066-4596-ae8e-4dbbe1d28ccc None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:37.003202 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b1bcac-1066-4596-ae8e-4dbbe1d28ccc None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:37.003480 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b1bcac-1066-4596-ae8e-4dbbe1d28ccc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:37.014090 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b1bcac-1066-4596-ae8e-4dbbe1d28ccc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:37.015582 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b1bcac-1066-4596-ae8e-4dbbe1d28ccc None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:37.031367 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-d2b1bcac-1066-4596-ae8e-4dbbe1d28ccc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:37.031367 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:37.031367 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:37.031367 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:37.031367 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:37.031367 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:37.031367 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:37.031367 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:37.031367 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:37.031367 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:37.031367 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:37.031367 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:37.031367 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:54:37.031367 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:54:37.031367 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:54:37.031367 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:54:37.032288 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:37.032288 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:37.032288 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:37.032288 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:37.032288 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:37.032288 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:37.032288 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:37.032288 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:37.032288 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:37.032288 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:37.032288 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:37.032288 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:37.032288 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:37.032288 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:37.032288 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:37.032288 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:37.032288 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:54:37.033735 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:54:37.033735 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:37.033735 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:37.033735 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 324/666] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:54:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:37.033735 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b1bcac-1066-4596-ae8e-4dbbe1d28ccc None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:37.103451 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b611af21-ca9a-4645-9d0e-e986c446bca5 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:37.139617 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b1bcac-1066-4596-ae8e-4dbbe1d28ccc None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:37.140146 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f083684-1e85-40a2-b3ed-191ed678cb3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:37.140756 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f083684-1e85-40a2-b3ed-191ed678cb3e None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:37.141152 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f083684-1e85-40a2-b3ed-191ed678cb3e None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:37.141438 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f083684-1e85-40a2-b3ed-191ed678cb3e None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:37.142214 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f083684-1e85-40a2-b3ed-191ed678cb3e None admin] RBAC: Adding query filter params (name=service) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:37.142516 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f083684-1e85-40a2-b3ed-191ed678cb3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:37.143929 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f083684-1e85-40a2-b3ed-191ed678cb3e None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:37.157701 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 343/667] 10.222.0.82 () {56 vars in 1103 bytes} [Thu Aug 18 16:54:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:37.157701 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f083684-1e85-40a2-b3ed-191ed678cb3e None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:37.179812 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae721ea5-3c98-4b16-9391-8d04fced60a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:37.180459 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae721ea5-3c98-4b16-9391-8d04fced60a2 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:37.180743 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae721ea5-3c98-4b16-9391-8d04fced60a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:37.181018 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae721ea5-3c98-4b16-9391-8d04fced60a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:37.191731 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae721ea5-3c98-4b16-9391-8d04fced60a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:37.193349 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae721ea5-3c98-4b16-9391-8d04fced60a2 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:37.202809 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:37.209820 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-ae721ea5-3c98-4b16-9391-8d04fced60a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:37.209820 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:37.209820 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:37.209820 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:37.209820 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:37.209820 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:37.209820 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:37.209820 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:37.209820 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:37.209820 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:37.209820 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:37.209820 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:37.209820 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:54:37.209820 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:54:37.209820 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:54:37.209820 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:54:37.210973 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:37.210973 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:37.210973 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:37.210973 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:37.210973 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:37.210973 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:37.210973 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:37.210973 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:37.210973 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:37.210973 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:37.210973 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:37.210973 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:37.210973 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:37.210973 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:37.210973 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:37.210973 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:37.210973 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:54:37.211942 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:54:37.211942 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:37.211942 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:37.211942 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 325/668] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:54:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:37.211942 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae721ea5-3c98-4b16-9391-8d04fced60a2 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:37.222836 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:37.267990 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f083684-1e85-40a2-b3ed-191ed678cb3e None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:37.307308 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eced54a-9361-4448-b572-f2c604d4742f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:37.308081 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eced54a-9361-4448-b572-f2c604d4742f None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:37.308681 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eced54a-9361-4448-b572-f2c604d4742f None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:37.308968 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eced54a-9361-4448-b572-f2c604d4742f None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:37.310069 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eced54a-9361-4448-b572-f2c604d4742f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:37.310381 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eced54a-9361-4448-b572-f2c604d4742f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:37.312610 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eced54a-9361-4448-b572-f2c604d4742f None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:37.317217 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae721ea5-3c98-4b16-9391-8d04fced60a2 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:37.332005 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 344/669] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:54:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:37.332005 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eced54a-9361-4448-b572-f2c604d4742f None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:37.356847 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e90972-e38d-40d2-ac38-32547cd4aa1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:37.357806 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e90972-e38d-40d2-ac38-32547cd4aa1e None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:37.358420 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e90972-e38d-40d2-ac38-32547cd4aa1e None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:37.358842 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e90972-e38d-40d2-ac38-32547cd4aa1e None admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:37.360215 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e90972-e38d-40d2-ac38-32547cd4aa1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:37.360673 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e90972-e38d-40d2-ac38-32547cd4aa1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:37.362836 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e90972-e38d-40d2-ac38-32547cd4aa1e None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:37.382430 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 326/670] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:54:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:37.382430 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e90972-e38d-40d2-ac38-32547cd4aa1e None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:37.445678 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eced54a-9361-4448-b572-f2c604d4742f None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:37.446513 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:37.464296 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:37.482023 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4ec2fd-4c69-4850-9eff-81fadfdf69cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:37.483019 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ec2fd-4c69-4850-9eff-81fadfdf69cb None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:37.483483 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ec2fd-4c69-4850-9eff-81fadfdf69cb None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:37.483765 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ec2fd-4c69-4850-9eff-81fadfdf69cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:37.490882 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e90972-e38d-40d2-ac38-32547cd4aa1e None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:37.493795 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4ec2fd-4c69-4850-9eff-81fadfdf69cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:37.495271 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4ec2fd-4c69-4850-9eff-81fadfdf69cb None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:37.510111 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-2b4ec2fd-4c69-4850-9eff-81fadfdf69cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:37.510111 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:37.510111 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:37.510111 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:37.510111 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:37.510111 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:37.510111 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:37.510111 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:37.510111 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:37.510111 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:37.510111 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:37.510111 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:37.510111 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:54:37.510111 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:54:37.510111 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:54:37.510111 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:54:37.511246 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:37.511246 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:37.511246 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:37.511246 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:37.511246 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:37.511246 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:37.511246 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:37.511246 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:37.511246 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:37.511246 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:37.511246 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:37.511246 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:37.511246 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:37.511246 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:37.511246 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:37.511246 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:37.511246 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:54:37.512990 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:54:37.512990 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:37.512990 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:37.512990 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 345/671] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:54:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:37.512990 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4ec2fd-4c69-4850-9eff-81fadfdf69cb None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:37.528789 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85bce66-7b5c-4927-8787-60ac99f32d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:37.529403 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85bce66-7b5c-4927-8787-60ac99f32d4a None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:37.529680 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85bce66-7b5c-4927-8787-60ac99f32d4a None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:37.529948 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85bce66-7b5c-4927-8787-60ac99f32d4a None admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:37.530586 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85bce66-7b5c-4927-8787-60ac99f32d4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:37.530882 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85bce66-7b5c-4927-8787-60ac99f32d4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:37.532274 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85bce66-7b5c-4927-8787-60ac99f32d4a None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:37.543862 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 327/672] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:54:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:37.544712 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85bce66-7b5c-4927-8787-60ac99f32d4a None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:37.622275 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4ec2fd-4c69-4850-9eff-81fadfdf69cb None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:37.645125 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85bce66-7b5c-4927-8787-60ac99f32d4a None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:37.655464 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4731268a-809f-4a60-b2cc-baa205c0cefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:37.656083 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4731268a-809f-4a60-b2cc-baa205c0cefe None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:37.656364 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4731268a-809f-4a60-b2cc-baa205c0cefe None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:37.656633 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4731268a-809f-4a60-b2cc-baa205c0cefe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:37.663253 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4731268a-809f-4a60-b2cc-baa205c0cefe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:37.664698 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4731268a-809f-4a60-b2cc-baa205c0cefe None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:37.665119 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:37.676016 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-4731268a-809f-4a60-b2cc-baa205c0cefe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:54:37.676016 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:37.676016 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:37.676016 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:37.676016 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:37.676016 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:37.676016 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:37.676016 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:37.676016 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:37.676016 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:37.676016 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:37.676016 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:37.676016 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 16:54:37.676016 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 16:54:37.676016 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 16:54:37.677210 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 16:54:37.677210 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:37.677210 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:37.677210 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:37.677210 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:37.677210 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:37.677210 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:37.677210 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:37.677210 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:37.677210 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:37.677210 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:37.677210 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:37.677210 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:37.677210 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:37.677210 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:37.677210 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:37.677210 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:37.678329 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 16:54:37.678329 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 16:54:37.678329 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 16:54:37.678329 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 16:54:37.678329 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 16:54:37.678329 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 16:54:37.678329 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:54:37.678329 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:37.678329 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 346/673] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 16:54:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:37.679094 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea877c7a-633a-4080-a4c5-1dc57f171d4a None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:37.679094 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea877c7a-633a-4080-a4c5-1dc57f171d4a None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:37.679094 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea877c7a-633a-4080-a4c5-1dc57f171d4a None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:37.680312 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 347/674] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:54:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:54:37.681782 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7612f6d7-c4d8-46e3-b51c-a4b1eb1628a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:37.682383 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea877c7a-633a-4080-a4c5-1dc57f171d4a None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:37.682613 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612f6d7-c4d8-46e3-b51c-a4b1eb1628a8 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:37.683021 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612f6d7-c4d8-46e3-b51c-a4b1eb1628a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:37.683388 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612f6d7-c4d8-46e3-b51c-a4b1eb1628a8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:37.690795 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7612f6d7-c4d8-46e3-b51c-a4b1eb1628a8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:37.692282 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7612f6d7-c4d8-46e3-b51c-a4b1eb1628a8 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:37.703808 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-7612f6d7-c4d8-46e3-b51c-a4b1eb1628a8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:54:37.703808 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:37.703808 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:37.703808 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:37.703808 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:37.703808 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:37.703808 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:37.703808 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:37.703808 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:37.703808 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:37.703808 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:37.703808 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:37.703808 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 16:54:37.703808 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 16:54:37.703808 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 16:54:37.704788 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 16:54:37.704788 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:37.704788 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:37.704788 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:37.704788 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:37.704788 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:37.704788 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:37.704788 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:37.704788 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:37.704788 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:37.704788 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:37.704788 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:37.704788 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:37.704788 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:37.704788 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:37.704788 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:37.704788 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:37.705908 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 16:54:37.705908 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 16:54:37.705908 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 16:54:37.705908 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 16:54:37.705908 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 16:54:37.705908 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 16:54:37.705908 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:54:37.705908 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:37.707031 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 328/675] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 16:54:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:37.708027 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7612f6d7-c4d8-46e3-b51c-a4b1eb1628a8 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:37.714212 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:37.786217 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea877c7a-633a-4080-a4c5-1dc57f171d4a None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:37.803402 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7612f6d7-c4d8-46e3-b51c-a4b1eb1628a8 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:37.819379 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbfa39c-0d31-4a2a-9b94-4347a7b0bcdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:37.820100 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbfa39c-0d31-4a2a-9b94-4347a7b0bcdb None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:37.820451 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbfa39c-0d31-4a2a-9b94-4347a7b0bcdb None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:37.820728 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbfa39c-0d31-4a2a-9b94-4347a7b0bcdb None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:37.821367 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbfa39c-0d31-4a2a-9b94-4347a7b0bcdb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:37.821669 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbfa39c-0d31-4a2a-9b94-4347a7b0bcdb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:37.823070 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbfa39c-0d31-4a2a-9b94-4347a7b0bcdb None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:37.834699 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 348/676] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:54:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:37.835521 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbfa39c-0d31-4a2a-9b94-4347a7b0bcdb None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:37.836871 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da566d5-ae4b-4735-9eac-cd63a24bca92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:37.837503 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da566d5-ae4b-4735-9eac-cd63a24bca92 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:37.837800 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da566d5-ae4b-4735-9eac-cd63a24bca92 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:37.838057 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da566d5-ae4b-4735-9eac-cd63a24bca92 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:37.845069 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da566d5-ae4b-4735-9eac-cd63a24bca92 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:37.846521 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da566d5-ae4b-4735-9eac-cd63a24bca92 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:37.857753 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-9da566d5-ae4b-4735-9eac-cd63a24bca92 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:54:37.857753 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:37.857753 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:37.857753 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:37.857753 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:37.857753 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:37.857753 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:37.857753 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:37.857753 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:37.857753 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:37.857753 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:37.857753 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:37.857753 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 16:54:37.857753 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 16:54:37.857753 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 16:54:37.858786 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 16:54:37.858786 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:37.858786 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:37.858786 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:37.858786 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:37.858786 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:37.858786 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:37.858786 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:37.858786 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:37.858786 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:37.858786 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:37.858786 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:37.858786 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:37.858786 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:37.858786 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:37.858786 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:37.858786 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:37.859906 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 16:54:37.859906 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 16:54:37.859906 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 16:54:37.859906 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 16:54:37.859906 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 16:54:37.859906 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 16:54:37.859906 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:54:37.859906 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:37.859906 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 329/677] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 16:54:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:37.860460 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b420df-e956-4a36-85e4-60f3005bc58e None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:37.860460 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b420df-e956-4a36-85e4-60f3005bc58e None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:37.860623 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b420df-e956-4a36-85e4-60f3005bc58e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:37.879456 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-88b420df-e956-4a36-85e4-60f3005bc58e None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:54:37.880604 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-88b420df-e956-4a36-85e4-60f3005bc58e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:54:37.914106 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:37.930040 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:37.939436 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbfa39c-0d31-4a2a-9b94-4347a7b0bcdb None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:37.945917 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 330/678] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:54:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:54:37.946741 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b420df-e956-4a36-85e4-60f3005bc58e None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:37.976879 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b803061d-a7d7-4cb4-b796-ab66aad88533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:37.977585 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b803061d-a7d7-4cb4-b796-ab66aad88533 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:37.978221 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b803061d-a7d7-4cb4-b796-ab66aad88533 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:37.978656 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b803061d-a7d7-4cb4-b796-ab66aad88533 None admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:37.979724 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b803061d-a7d7-4cb4-b796-ab66aad88533 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:37.980034 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b803061d-a7d7-4cb4-b796-ab66aad88533 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:37.981900 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b803061d-a7d7-4cb4-b796-ab66aad88533 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:37.999491 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 349/679] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 16:54:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:37.999491 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b803061d-a7d7-4cb4-b796-ab66aad88533 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:38.068763 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b420df-e956-4a36-85e4-60f3005bc58e None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:38.098883 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b803061d-a7d7-4cb4-b796-ab66aad88533 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:38.103175 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67878459-ed14-4787-ab91-a39819adb142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:38.103785 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67878459-ed14-4787-ab91-a39819adb142 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:38.104063 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67878459-ed14-4787-ab91-a39819adb142 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:38.104331 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67878459-ed14-4787-ab91-a39819adb142 None admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:38.105018 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67878459-ed14-4787-ab91-a39819adb142 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:38.105322 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67878459-ed14-4787-ab91-a39819adb142 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:38.107696 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67878459-ed14-4787-ab91-a39819adb142 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:38.119675 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 331/680] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 16:54:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:38.120809 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67878459-ed14-4787-ab91-a39819adb142 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:38.140916 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbd8c3d-4553-4c9d-bd0c-fd07ee749914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:38.142100 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd8c3d-4553-4c9d-bd0c-fd07ee749914 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:38.142377 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd8c3d-4553-4c9d-bd0c-fd07ee749914 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:38.142642 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd8c3d-4553-4c9d-bd0c-fd07ee749914 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:38.150500 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbd8c3d-4553-4c9d-bd0c-fd07ee749914 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:38.151965 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbd8c3d-4553-4c9d-bd0c-fd07ee749914 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:38.153212 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:38.164157 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-7bbd8c3d-4553-4c9d-bd0c-fd07ee749914 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 18 16:54:38.164157 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:38.164157 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:38.164157 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:38.164157 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:38.164157 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:38.164157 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:38.164157 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:38.164157 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:38.164157 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:38.164157 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:38.164157 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:38.164157 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 16:54:38.164157 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 16:54:38.164157 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 16:54:38.164157 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 16:54:38.165077 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:38.165077 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:38.165077 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 16:54:38.165077 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:54:38.165077 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 16:54:38.165077 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:54:38.165077 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:38.165077 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:38.165077 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:38.165077 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:38.165077 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:38.165077 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:38.165077 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:38.165077 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:38.165077 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:38.165077 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:38.165077 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:38.165077 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:38.171045 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:38.171045 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:38.171045 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 16:54:38.171045 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 16:54:38.171045 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 16:54:38.171045 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 16:54:38.171045 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 16:54:38.171045 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 16:54:38.171045 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 18 16:54:38.171045 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:38.171045 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 350/681] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 16:54:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:38.171045 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbd8c3d-4553-4c9d-bd0c-fd07ee749914 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:38.178824 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:38.235090 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67878459-ed14-4787-ab91-a39819adb142 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:38.268780 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d90f91-6678-4250-a700-8a49ec07481a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:38.269359 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbd8c3d-4553-4c9d-bd0c-fd07ee749914 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:38.269466 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d90f91-6678-4250-a700-8a49ec07481a None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:38.269716 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d90f91-6678-4250-a700-8a49ec07481a None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:38.269993 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d90f91-6678-4250-a700-8a49ec07481a None admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:38.270707 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d90f91-6678-4250-a700-8a49ec07481a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:38.271046 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d90f91-6678-4250-a700-8a49ec07481a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:38.272491 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d90f91-6678-4250-a700-8a49ec07481a None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:38.285198 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 332/682] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 16:54:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:38.285991 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d90f91-6678-4250-a700-8a49ec07481a None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:38.303480 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bba4ce5-0d28-4000-a279-0438d3f503d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:38.304075 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba4ce5-0d28-4000-a279-0438d3f503d7 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:38.304353 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba4ce5-0d28-4000-a279-0438d3f503d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:38.304625 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba4ce5-0d28-4000-a279-0438d3f503d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:38.305335 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bba4ce5-0d28-4000-a279-0438d3f503d7 None admin] RBAC: Adding query filter params (role.id=5e0f402e1af84b44a723ee0730315d74, scope.project.id=2293c0aab61f44849d0d48b214cde7ea, user.id=2bee1ba6bb324de4915904f83349b3c4) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:38.305637 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bba4ce5-0d28-4000-a279-0438d3f503d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:38.307077 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bba4ce5-0d28-4000-a279-0438d3f503d7 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:38.322948 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 351/683] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 16:54:38 2022] GET /identity/v3/role_assignments?user.id=2bee1ba6bb324de4915904f83349b3c4&scope.project.id=2293c0aab61f44849d0d48b214cde7ea&role.id=5e0f402e1af84b44a723ee0730315d74 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:38.322948 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bba4ce5-0d28-4000-a279-0438d3f503d7 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:38.384778 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:38.395891 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d90f91-6678-4250-a700-8a49ec07481a None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:38.403986 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:38.430145 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bba4ce5-0d28-4000-a279-0438d3f503d7 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:38.436274 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e29be5-db31-43a8-8a68-90893fe2bc8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:38.437061 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e29be5-db31-43a8-8a68-90893fe2bc8a None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:38.437418 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e29be5-db31-43a8-8a68-90893fe2bc8a None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:38.437785 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e29be5-db31-43a8-8a68-90893fe2bc8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:38.438634 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e29be5-db31-43a8-8a68-90893fe2bc8a None admin] RBAC: Adding query filter params (role.id=5e0f402e1af84b44a723ee0730315d74, scope.project.id=2293c0aab61f44849d0d48b214cde7ea, user.id=fe2403ede53148b2a113ffa2a605e987) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:38.439048 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e29be5-db31-43a8-8a68-90893fe2bc8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:38.440533 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e29be5-db31-43a8-8a68-90893fe2bc8a None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:38.447406 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 333/684] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 16:54:38 2022] GET /identity/v3/role_assignments?user.id=fe2403ede53148b2a113ffa2a605e987&scope.project.id=2293c0aab61f44849d0d48b214cde7ea&role.id=5e0f402e1af84b44a723ee0730315d74 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:38.454462 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e29be5-db31-43a8-8a68-90893fe2bc8a None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:38.487021 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c7fc5f-1658-4595-a49f-caa695052272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:38.488524 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c7fc5f-1658-4595-a49f-caa695052272 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:38.488524 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c7fc5f-1658-4595-a49f-caa695052272 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:38.488524 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c7fc5f-1658-4595-a49f-caa695052272 None admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:38.488929 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c7fc5f-1658-4595-a49f-caa695052272 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:38.489238 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c7fc5f-1658-4595-a49f-caa695052272 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:38.491919 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c7fc5f-1658-4595-a49f-caa695052272 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:38.517437 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 352/685] 10.222.0.82 () {56 vars in 1135 bytes} [Thu Aug 18 16:54:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:38.523003 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c7fc5f-1658-4595-a49f-caa695052272 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:38.610010 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e29be5-db31-43a8-8a68-90893fe2bc8a None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:38.656576 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c7fc5f-1658-4595-a49f-caa695052272 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:38.658785 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b87780-e8a1-4d2e-bcec-5d5d7128fed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:38.659468 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b87780-e8a1-4d2e-bcec-5d5d7128fed2 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:38.659758 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b87780-e8a1-4d2e-bcec-5d5d7128fed2 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:38.660034 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b87780-e8a1-4d2e-bcec-5d5d7128fed2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:38.675740 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b87780-e8a1-4d2e-bcec-5d5d7128fed2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:54:38.680769 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-11b87780-e8a1-4d2e-bcec-5d5d7128fed2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 16:54:38.680769 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:38.680769 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:38.680769 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:38.680769 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:38.680769 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:38.680769 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:38.680769 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:38.680769 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:38.680769 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:38.680769 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:38.680769 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:38.680769 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:54:38.680769 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:54:38.680769 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:54:38.680769 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:54:38.681702 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:54:38.681702 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:54:38.681702 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:54:38.681702 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:54:38.681702 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:38.681702 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:38.681702 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:38.681702 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:38.681702 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:38.681702 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:38.681702 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:38.681702 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:38.681702 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:38.681702 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:38.681702 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:38.681702 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:38.682521 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:38.682521 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:38.682521 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:38.682521 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:38.682521 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:54:38.682521 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:54:38.682521 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:54:38.682521 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:54:38.682521 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:54:38.682521 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:54:38.682521 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 16:54:38.682521 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:38.682521 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 334/686] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 16:54:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:38.683286 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b87780-e8a1-4d2e-bcec-5d5d7128fed2 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:38.701974 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:38.702662 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:38.708861 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b0d35-686b-4ddc-9936-4ee86e7cf23d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:38.709626 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b0d35-686b-4ddc-9936-4ee86e7cf23d None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:38.710037 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b0d35-686b-4ddc-9936-4ee86e7cf23d None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:38.710380 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b0d35-686b-4ddc-9936-4ee86e7cf23d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:38.711166 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804b0d35-686b-4ddc-9936-4ee86e7cf23d None admin] RBAC: Adding query filter params (role.id=5e0f402e1af84b44a723ee0730315d74, scope.project.id=2293c0aab61f44849d0d48b214cde7ea, user.id=85876a26a22e4c2c882883f7e978e4f1) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:38.711478 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804b0d35-686b-4ddc-9936-4ee86e7cf23d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:38.716267 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804b0d35-686b-4ddc-9936-4ee86e7cf23d None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:38.719829 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 353/687] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 16:54:38 2022] GET /identity/v3/role_assignments?user.id=85876a26a22e4c2c882883f7e978e4f1&scope.project.id=2293c0aab61f44849d0d48b214cde7ea&role.id=5e0f402e1af84b44a723ee0730315d74 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:38.720660 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b0d35-686b-4ddc-9936-4ee86e7cf23d None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:38.866401 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b87780-e8a1-4d2e-bcec-5d5d7128fed2 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:38.911785 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59de4e10-8b03-4433-9f78-af946018e026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:38.914149 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59de4e10-8b03-4433-9f78-af946018e026 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:38.914149 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59de4e10-8b03-4433-9f78-af946018e026 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:38.914149 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59de4e10-8b03-4433-9f78-af946018e026 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:38.929544 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b0d35-686b-4ddc-9936-4ee86e7cf23d None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:38.979197 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9236e1e-fd34-4553-8ac9-025b0bbc9d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:38.980367 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9236e1e-fd34-4553-8ac9-025b0bbc9d9b None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:38.980810 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9236e1e-fd34-4553-8ac9-025b0bbc9d9b None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:38.981706 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9236e1e-fd34-4553-8ac9-025b0bbc9d9b None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:38.983123 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9236e1e-fd34-4553-8ac9-025b0bbc9d9b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:38.983717 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9236e1e-fd34-4553-8ac9-025b0bbc9d9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:38.985625 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9236e1e-fd34-4553-8ac9-025b0bbc9d9b None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:38.997657 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 354/688] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:54:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:39.002980 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9236e1e-fd34-4553-8ac9-025b0bbc9d9b None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:39.014562 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59de4e10-8b03-4433-9f78-af946018e026 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:39.016106 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59de4e10-8b03-4433-9f78-af946018e026 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:39.039556 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:39.082534 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:39.130617 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9236e1e-fd34-4553-8ac9-025b0bbc9d9b None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:39.134069 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 335/689] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:54:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 16:54:39.139599 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59de4e10-8b03-4433-9f78-af946018e026 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:39.171019 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04641da-2c42-42da-9eba-fac86443e82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:39.172313 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04641da-2c42-42da-9eba-fac86443e82b None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:39.172860 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04641da-2c42-42da-9eba-fac86443e82b None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:39.173333 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04641da-2c42-42da-9eba-fac86443e82b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:39.185848 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04641da-2c42-42da-9eba-fac86443e82b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:39.187761 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04641da-2c42-42da-9eba-fac86443e82b None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:39.204279 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-e04641da-2c42-42da-9eba-fac86443e82b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:39.204279 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:39.204279 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:39.204279 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:39.204279 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:39.204279 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:39.204279 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:39.204279 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:39.204279 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:39.204279 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:39.204279 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:39.204279 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:39.204279 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:54:39.204279 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:54:39.204279 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:54:39.204279 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:54:39.205298 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:39.205298 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:39.205298 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:39.205298 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:39.205298 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:39.205298 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:39.205298 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:39.205298 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:39.205298 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:39.205298 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:39.205298 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:39.205298 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:39.205298 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:39.205298 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:39.205298 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:39.205298 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:39.205298 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:54:39.206178 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:54:39.206178 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:39.206178 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:39.208438 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 355/690] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:54:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 16:54:39.219967 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04641da-2c42-42da-9eba-fac86443e82b None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:39.242321 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59de4e10-8b03-4433-9f78-af946018e026 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:39.258734 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:39.280347 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545aea93-8df9-4dad-9ea1-da97b87d9182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:39.280995 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545aea93-8df9-4dad-9ea1-da97b87d9182 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:39.281277 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545aea93-8df9-4dad-9ea1-da97b87d9182 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:39.281547 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545aea93-8df9-4dad-9ea1-da97b87d9182 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:39.292095 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545aea93-8df9-4dad-9ea1-da97b87d9182 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:39.294492 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545aea93-8df9-4dad-9ea1-da97b87d9182 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:39.310980 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-545aea93-8df9-4dad-9ea1-da97b87d9182 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:39.310980 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:39.310980 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:39.310980 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:39.310980 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:39.310980 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:39.310980 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:39.310980 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:39.310980 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:39.310980 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:39.310980 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:39.310980 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:39.310980 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:54:39.310980 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:54:39.310980 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:54:39.310980 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:54:39.311909 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:39.311909 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:39.311909 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:39.311909 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:39.311909 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:39.311909 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:39.311909 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:39.311909 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:39.311909 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:39.311909 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:39.311909 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:39.311909 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:39.311909 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:39.311909 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:39.311909 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:39.311909 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:39.311909 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:54:39.312866 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:54:39.312866 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:39.312866 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:39.319322 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 336/691] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:54:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 16:54:39.324844 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545aea93-8df9-4dad-9ea1-da97b87d9182 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:39.352315 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04641da-2c42-42da-9eba-fac86443e82b None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:39.364595 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:39.389723 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6b2082-9976-487a-b1ff-47659fc8db0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:39.390704 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6b2082-9976-487a-b1ff-47659fc8db0d None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:39.391383 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6b2082-9976-487a-b1ff-47659fc8db0d None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:39.392066 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6b2082-9976-487a-b1ff-47659fc8db0d None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:39.392997 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6b2082-9976-487a-b1ff-47659fc8db0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:39.393495 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6b2082-9976-487a-b1ff-47659fc8db0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:39.395182 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6b2082-9976-487a-b1ff-47659fc8db0d None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:39.409595 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 356/692] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:54:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:39.415535 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6b2082-9976-487a-b1ff-47659fc8db0d None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:39.535159 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6b2082-9976-487a-b1ff-47659fc8db0d None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:39.535371 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545aea93-8df9-4dad-9ea1-da97b87d9182 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:39.573005 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b1277d-4a25-4fda-ad52-80e6c59182ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:39.573791 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b1277d-4a25-4fda-ad52-80e6c59182ca None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:39.574179 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b1277d-4a25-4fda-ad52-80e6c59182ca None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:39.574553 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b1277d-4a25-4fda-ad52-80e6c59182ca None admin] PATH_INFO: `/v3/users/nova` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:39.582754 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b1277d-4a25-4fda-ad52-80e6c59182ca None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:39.584352 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b1277d-4a25-4fda-ad52-80e6c59182ca None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:39.597842 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-35b1277d-4a25-4fda-ad52-80e6c59182ca None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 18 16:54:39.597842 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:39.597842 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:39.597842 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:39.597842 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:39.597842 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:39.597842 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:39.597842 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:39.597842 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:39.597842 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:39.597842 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:39.597842 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:39.597842 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 16:54:39.597842 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 16:54:39.597842 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 16:54:39.597842 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 16:54:39.599038 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:39.599038 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:39.599038 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 16:54:39.599038 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:54:39.599038 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 16:54:39.599038 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:54:39.599038 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:39.599038 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:39.599038 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:39.599038 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:39.599038 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:39.599038 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:39.599038 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:39.599038 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:39.599038 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:39.599038 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:39.599038 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:39.599038 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:39.600218 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:39.600218 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:39.600218 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 16:54:39.600218 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 16:54:39.600218 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 16:54:39.600218 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 16:54:39.600218 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 16:54:39.600218 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 16:54:39.600218 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 18 16:54:39.600218 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:39.601931 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 357/693] 10.222.0.82 () {56 vars in 1093 bytes} [Thu Aug 18 16:54:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:39.607674 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b1277d-4a25-4fda-ad52-80e6c59182ca None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:39.621052 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3d9e48-0cfe-463d-80c2-64c043bd7787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:39.622464 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3d9e48-0cfe-463d-80c2-64c043bd7787 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:39.622852 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3d9e48-0cfe-463d-80c2-64c043bd7787 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:39.623254 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3d9e48-0cfe-463d-80c2-64c043bd7787 None admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:39.626452 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3d9e48-0cfe-463d-80c2-64c043bd7787 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:39.626877 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3d9e48-0cfe-463d-80c2-64c043bd7787 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:39.628398 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3d9e48-0cfe-463d-80c2-64c043bd7787 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:39.652737 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:39.655458 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 337/694] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:54:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:54:39.660499 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3d9e48-0cfe-463d-80c2-64c043bd7787 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:39.691279 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:39.733612 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b1277d-4a25-4fda-ad52-80e6c59182ca None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:39.774731 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a0bca-d6c2-40d2-a844-3dab43e5e590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:39.775524 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a0bca-d6c2-40d2-a844-3dab43e5e590 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:39.775972 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a0bca-d6c2-40d2-a844-3dab43e5e590 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:39.776350 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a0bca-d6c2-40d2-a844-3dab43e5e590 None admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:39.777209 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977a0bca-d6c2-40d2-a844-3dab43e5e590 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:39.777618 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977a0bca-d6c2-40d2-a844-3dab43e5e590 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:39.778756 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3d9e48-0cfe-463d-80c2-64c043bd7787 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:39.782087 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977a0bca-d6c2-40d2-a844-3dab43e5e590 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:39.801802 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 358/695] 10.222.0.82 () {56 vars in 1133 bytes} [Thu Aug 18 16:54:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:54:39.812366 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a0bca-d6c2-40d2-a844-3dab43e5e590 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:39.830439 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc39aa9a-3a28-40e1-a77e-450aaa7b5512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:39.831748 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39aa9a-3a28-40e1-a77e-450aaa7b5512 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:39.832225 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39aa9a-3a28-40e1-a77e-450aaa7b5512 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:39.832677 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39aa9a-3a28-40e1-a77e-450aaa7b5512 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:39.842535 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc39aa9a-3a28-40e1-a77e-450aaa7b5512 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:39.844380 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc39aa9a-3a28-40e1-a77e-450aaa7b5512 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:39.857526 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-fc39aa9a-3a28-40e1-a77e-450aaa7b5512 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:54:39.857526 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:39.857526 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:39.857526 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:39.857526 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:39.857526 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:39.857526 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:39.857526 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:39.857526 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:39.857526 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:39.857526 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:39.857526 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:39.857526 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 16:54:39.857526 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 16:54:39.857526 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 16:54:39.858573 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 16:54:39.858573 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:39.858573 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:39.858573 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:39.858573 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:39.858573 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:39.858573 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:39.858573 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:39.858573 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:39.858573 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:39.858573 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:39.858573 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:39.858573 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:39.858573 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:39.858573 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:39.858573 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:39.858573 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:39.859677 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 16:54:39.859677 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 16:54:39.859677 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 16:54:39.859677 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 16:54:39.859677 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 16:54:39.859677 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 16:54:39.859677 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:54:39.859677 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:39.861784 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 338/696] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 16:54:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 16:54:39.867841 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc39aa9a-3a28-40e1-a77e-450aaa7b5512 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:39.985468 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:40.011897 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a0bca-d6c2-40d2-a844-3dab43e5e590 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:40.027302 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:40.054970 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc39aa9a-3a28-40e1-a77e-450aaa7b5512 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:40.081042 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8c0fd5-a855-41bc-b25e-c78166fc2621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:40.081723 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8c0fd5-a855-41bc-b25e-c78166fc2621 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:40.082006 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8c0fd5-a855-41bc-b25e-c78166fc2621 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:40.082277 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8c0fd5-a855-41bc-b25e-c78166fc2621 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:40.124042 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a214bb90-6b3f-4b2d-b1b8-bfdec9a7083e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:40.124667 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214bb90-6b3f-4b2d-b1b8-bfdec9a7083e None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:40.124949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214bb90-6b3f-4b2d-b1b8-bfdec9a7083e None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:40.125227 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214bb90-6b3f-4b2d-b1b8-bfdec9a7083e None admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:40.125922 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a214bb90-6b3f-4b2d-b1b8-bfdec9a7083e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:40.126229 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a214bb90-6b3f-4b2d-b1b8-bfdec9a7083e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:40.128962 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a214bb90-6b3f-4b2d-b1b8-bfdec9a7083e None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:40.157998 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 339/697] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 16:54:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:40.167494 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a214bb90-6b3f-4b2d-b1b8-bfdec9a7083e None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:40.203777 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8c0fd5-a855-41bc-b25e-c78166fc2621 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:40.205255 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8c0fd5-a855-41bc-b25e-c78166fc2621 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:40.305391 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a214bb90-6b3f-4b2d-b1b8-bfdec9a7083e None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:40.347531 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c0cce7-7eff-4143-bae7-a8c7d4f22fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:40.348149 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c0cce7-7eff-4143-bae7-a8c7d4f22fa6 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:40.356377 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c0cce7-7eff-4143-bae7-a8c7d4f22fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:40.356377 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c0cce7-7eff-4143-bae7-a8c7d4f22fa6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:40.356377 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c0cce7-7eff-4143-bae7-a8c7d4f22fa6 None admin] RBAC: Adding query filter params (role.id=5e0f402e1af84b44a723ee0730315d74, scope.project.id=2293c0aab61f44849d0d48b214cde7ea, user.id=1381673f81584f4b8084823e3aa99e2a) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:40.356377 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c0cce7-7eff-4143-bae7-a8c7d4f22fa6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:40.356377 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c0cce7-7eff-4143-bae7-a8c7d4f22fa6 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:40.359273 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 340/698] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 16:54:40 2022] GET /identity/v3/role_assignments?user.id=1381673f81584f4b8084823e3aa99e2a&scope.project.id=2293c0aab61f44849d0d48b214cde7ea&role.id=5e0f402e1af84b44a723ee0730315d74 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:40.392160 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:40.414265 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 359/699] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:54:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 603 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 16:54:40.420700 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c0cce7-7eff-4143-bae7-a8c7d4f22fa6 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:40.433540 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:40.534817 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c0cce7-7eff-4143-bae7-a8c7d4f22fa6 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:40.572091 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029ba6ac-45fe-41f0-ac8e-e2de519e82b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:40.572672 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ba6ac-45fe-41f0-ac8e-e2de519e82b6 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:40.572955 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ba6ac-45fe-41f0-ac8e-e2de519e82b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:40.573232 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ba6ac-45fe-41f0-ac8e-e2de519e82b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:40.585168 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029ba6ac-45fe-41f0-ac8e-e2de519e82b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:40.586674 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029ba6ac-45fe-41f0-ac8e-e2de519e82b6 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:40.603241 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-029ba6ac-45fe-41f0-ac8e-e2de519e82b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:40.603241 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:40.603241 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:40.603241 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:40.603241 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:40.603241 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:40.603241 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:40.603241 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:40.603241 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:40.603241 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:40.603241 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:40.603241 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:40.603241 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:54:40.603241 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:54:40.603241 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:54:40.603241 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:54:40.604214 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:40.604214 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:40.604214 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:40.604214 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:40.604214 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:40.604214 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:40.604214 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:40.604214 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:40.604214 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:40.604214 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:40.604214 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:40.604214 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:40.604214 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:40.604214 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:40.604214 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:40.604214 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:40.604214 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:54:40.605071 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:54:40.605071 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:40.605071 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:40.605071 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 341/700] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:54:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:40.616064 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8c0fd5-a855-41bc-b25e-c78166fc2621 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:40.731793 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8c0fd5-a855-41bc-b25e-c78166fc2621 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:40.766849 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c40822-8a8f-4626-9bfc-676feaf6f157 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:40.767580 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c40822-8a8f-4626-9bfc-676feaf6f157 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:40.767580 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c40822-8a8f-4626-9bfc-676feaf6f157 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:40.769097 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 342/701] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:54:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 16:54:40.772709 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f18646c-e93e-49c9-8d5f-de646e197a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:40.773498 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18646c-e93e-49c9-8d5f-de646e197a67 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:40.773927 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18646c-e93e-49c9-8d5f-de646e197a67 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:40.774348 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18646c-e93e-49c9-8d5f-de646e197a67 None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:40.774461 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c790b567-9e7e-41ee-b9a3-daf5d6aef97b None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:40.774645 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c790b567-9e7e-41ee-b9a3-daf5d6aef97b None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:40.774954 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c790b567-9e7e-41ee-b9a3-daf5d6aef97b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:40.775590 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f18646c-e93e-49c9-8d5f-de646e197a67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:40.776040 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f18646c-e93e-49c9-8d5f-de646e197a67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:40.778277 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f18646c-e93e-49c9-8d5f-de646e197a67 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:40.792509 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 360/702] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:54:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:40.794392 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae43d6-8558-4342-bd1b-5560cf57518d None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:40.794691 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae43d6-8558-4342-bd1b-5560cf57518d None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:40.795029 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae43d6-8558-4342-bd1b-5560cf57518d None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:40.798992 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 361/703] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:54:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:54:40.799783 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afae43d6-8558-4342-bd1b-5560cf57518d None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:40.801335 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-c790b567-9e7e-41ee-b9a3-daf5d6aef97b None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:54:40.802567 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-c790b567-9e7e-41ee-b9a3-daf5d6aef97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:54:40.864590 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:40.876857 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:40.889561 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 343/704] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:54:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 16:54:40.890921 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc13252-8a67-4b30-ad79-9c625a0876df None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:40.891257 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc13252-8a67-4b30-ad79-9c625a0876df None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:40.891575 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc13252-8a67-4b30-ad79-9c625a0876df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:40.918360 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-9bc13252-8a67-4b30-ad79-9c625a0876df None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:54:40.920210 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-9bc13252-8a67-4b30-ad79-9c625a0876df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:54:40.961737 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afae43d6-8558-4342-bd1b-5560cf57518d None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:41.014870 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc1b2ea-ad41-4ae1-a1f3-74079cfff3c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:41.015544 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1b2ea-ad41-4ae1-a1f3-74079cfff3c3 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:41.015868 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1b2ea-ad41-4ae1-a1f3-74079cfff3c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:41.016147 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1b2ea-ad41-4ae1-a1f3-74079cfff3c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:41.027042 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc1b2ea-ad41-4ae1-a1f3-74079cfff3c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:41.027042 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc1b2ea-ad41-4ae1-a1f3-74079cfff3c3 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:41.041477 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-7dc1b2ea-ad41-4ae1-a1f3-74079cfff3c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:54:41.041477 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:41.041477 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:41.041477 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:41.041477 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:41.041477 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:41.041477 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:41.041477 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:41.041477 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:41.041477 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:41.041477 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:41.041477 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:41.041477 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 16:54:41.041477 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 16:54:41.041477 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 16:54:41.042410 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 16:54:41.042410 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:41.042410 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:41.042410 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:41.042410 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:41.042410 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:41.042410 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:41.042410 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:41.042410 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:41.042410 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:41.042410 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:41.042410 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:41.042410 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:41.042410 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:41.042410 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:41.042410 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:41.042410 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:41.043950 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 16:54:41.043950 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 16:54:41.043950 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 16:54:41.043950 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 16:54:41.043950 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 16:54:41.043950 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 16:54:41.043950 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:54:41.043950 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:41.043950 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 362/705] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 16:54:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:41.046083 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc1b2ea-ad41-4ae1-a1f3-74079cfff3c3 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:41.057228 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 344/706] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:54:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:54:41.151695 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc1b2ea-ad41-4ae1-a1f3-74079cfff3c3 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:41.162252 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:41.188172 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe71f11-378d-4450-ad3e-7a7f813bfb43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:41.188781 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe71f11-378d-4450-ad3e-7a7f813bfb43 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:41.189091 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe71f11-378d-4450-ad3e-7a7f813bfb43 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:41.196611 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe71f11-378d-4450-ad3e-7a7f813bfb43 None admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:41.196611 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe71f11-378d-4450-ad3e-7a7f813bfb43 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:41.196611 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe71f11-378d-4450-ad3e-7a7f813bfb43 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:41.196611 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe71f11-378d-4450-ad3e-7a7f813bfb43 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:41.205731 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 363/707] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 16:54:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:41.212014 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc13252-8a67-4b30-ad79-9c625a0876df None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:41.232935 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8046faa-d0b6-4f4a-85ce-189e7c128f3a None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:41.233530 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8046faa-d0b6-4f4a-85ce-189e7c128f3a None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:41.233657 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8046faa-d0b6-4f4a-85ce-189e7c128f3a None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:41.235246 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 364/708] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:54:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 16:54:41.240134 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b39f3b-1c5d-4ee1-af9a-c7e60e37647c None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:41.240598 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b39f3b-1c5d-4ee1-af9a-c7e60e37647c None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:41.240720 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b39f3b-1c5d-4ee1-af9a-c7e60e37647c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:41.258533 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-78b39f3b-1c5d-4ee1-af9a-c7e60e37647c None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:54:41.259765 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-78b39f3b-1c5d-4ee1-af9a-c7e60e37647c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:54:41.329872 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 365/709] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:54:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:54:41.335056 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b39f3b-1c5d-4ee1-af9a-c7e60e37647c None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:41.374059 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc13252-8a67-4b30-ad79-9c625a0876df None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:41.417449 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:41.442940 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d89352-7484-4326-a1ce-29cd49605f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:41.443887 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d89352-7484-4326-a1ce-29cd49605f97 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:41.444353 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d89352-7484-4326-a1ce-29cd49605f97 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:41.444839 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d89352-7484-4326-a1ce-29cd49605f97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:41.445759 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d89352-7484-4326-a1ce-29cd49605f97 None admin] RBAC: Adding query filter params (role.id=2c6966cfb10f4d72b0242e9fb5e13b17, scope.project.id=2293c0aab61f44849d0d48b214cde7ea, user.id=55723c756cae4e7ab636054db48de02d) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:41.446263 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d89352-7484-4326-a1ce-29cd49605f97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:41.449000 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d89352-7484-4326-a1ce-29cd49605f97 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:41.461255 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 345/710] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 16:54:41 2022] GET /identity/v3/role_assignments?user.id=55723c756cae4e7ab636054db48de02d&scope.project.id=2293c0aab61f44849d0d48b214cde7ea&role.id=2c6966cfb10f4d72b0242e9fb5e13b17 => generated 259 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:41.470018 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d89352-7484-4326-a1ce-29cd49605f97 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:41.495181 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:41.500652 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b39f3b-1c5d-4ee1-af9a-c7e60e37647c None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:41.538307 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b3da25-5a82-4760-8911-7f523c08f715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:41.539266 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b3da25-5a82-4760-8911-7f523c08f715 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:41.539725 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b3da25-5a82-4760-8911-7f523c08f715 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:41.540143 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b3da25-5a82-4760-8911-7f523c08f715 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:41.551705 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b3da25-5a82-4760-8911-7f523c08f715 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:41.553531 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b3da25-5a82-4760-8911-7f523c08f715 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:41.570154 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-a4b3da25-5a82-4760-8911-7f523c08f715 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:41.570154 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:41.570154 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:41.570154 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:41.570154 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:41.570154 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:41.570154 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:41.570154 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:41.570154 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:41.570154 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:41.570154 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:41.570154 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:41.570154 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:54:41.570154 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:54:41.570154 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:54:41.570154 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:54:41.571229 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:41.571229 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:41.571229 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:41.571229 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:41.571229 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:41.571229 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:41.571229 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:41.571229 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:41.571229 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:41.571229 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:41.571229 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:41.571229 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:41.571229 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:41.571229 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:41.571229 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:41.571229 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:41.571229 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:54:41.572085 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:54:41.572085 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:41.572085 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:41.574312 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 366/711] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:54:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:41.582956 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b3da25-5a82-4760-8911-7f523c08f715 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:41.603916 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d89352-7484-4326-a1ce-29cd49605f97 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:41.674233 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2907073-c733-4e04-ae52-416842f00abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:41.674846 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2907073-c733-4e04-ae52-416842f00abf None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:41.675160 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2907073-c733-4e04-ae52-416842f00abf None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:41.675440 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2907073-c733-4e04-ae52-416842f00abf None admin] PATH_INFO: `/v3/services/network` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:41.676029 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2907073-c733-4e04-ae52-416842f00abf None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:41.677416 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2907073-c733-4e04-ae52-416842f00abf None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:41.696595 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-e2907073-c733-4e04-ae52-416842f00abf None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 18 16:54:41.696595 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:41.696595 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 16:54:41.696595 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 16:54:41.696595 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 16:54:41.696595 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 16:54:41.696595 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 16:54:41.696595 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 16:54:41.696595 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 18 16:54:41.696595 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:41.696595 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 16:54:41.696595 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:41.696595 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:41.696595 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:41.696595 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:41.696595 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:41.696595 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:41.697834 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:41.697834 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:41.697834 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:41.697834 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:41.697834 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:41.697834 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:41.697834 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 16:54:41.697834 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 16:54:41.697834 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 16:54:41.697834 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 16:54:41.697834 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:41.697834 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:41.697834 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:41.697834 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:41.697834 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:41.697834 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:41.698761 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:41.698761 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:41.698761 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:41.698761 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:41.698761 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:41.698761 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:41.698761 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:41.698761 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:41.698761 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:41.698761 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:41.698761 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 16:54:41.698761 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 16:54:41.698761 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 18 16:54:41.698761 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:41.698761 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 346/712] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 16:54:41 2022] GET /identity/v3/services/network => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 16:54:41.700267 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2907073-c733-4e04-ae52-416842f00abf None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:41.764334 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:41.769556 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b3da25-5a82-4760-8911-7f523c08f715 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:41.808296 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff45526-f9ad-4dde-841f-7d88d34cb521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:41.809316 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff45526-f9ad-4dde-841f-7d88d34cb521 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:41.809671 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff45526-f9ad-4dde-841f-7d88d34cb521 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:41.810066 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff45526-f9ad-4dde-841f-7d88d34cb521 None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:41.810819 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff45526-f9ad-4dde-841f-7d88d34cb521 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:41.811275 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff45526-f9ad-4dde-841f-7d88d34cb521 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:41.812837 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff45526-f9ad-4dde-841f-7d88d34cb521 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:41.826018 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 367/713] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:54:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:41.830777 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff45526-f9ad-4dde-841f-7d88d34cb521 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:41.839117 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:41.841970 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2907073-c733-4e04-ae52-416842f00abf None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:41.879472 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8a1cd5-7189-42fa-9fbe-c114dc252d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:41.880453 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a1cd5-7189-42fa-9fbe-c114dc252d49 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:41.880922 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a1cd5-7189-42fa-9fbe-c114dc252d49 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:41.881468 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a1cd5-7189-42fa-9fbe-c114dc252d49 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:41.882235 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8a1cd5-7189-42fa-9fbe-c114dc252d49 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:41.883861 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8a1cd5-7189-42fa-9fbe-c114dc252d49 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:41.895179 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-9e8a1cd5-7189-42fa-9fbe-c114dc252d49 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 18 16:54:41.895179 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:41.895179 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 16:54:41.895179 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 16:54:41.895179 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 16:54:41.895179 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 16:54:41.895179 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 16:54:41.895179 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 16:54:41.895179 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 18 16:54:41.895179 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:41.895179 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 16:54:41.895179 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:41.895179 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:41.895179 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:41.895179 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:41.895179 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:41.895179 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:41.896338 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:41.896338 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:41.896338 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:41.896338 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:41.896338 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:41.896338 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:41.896338 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 16:54:41.896338 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 16:54:41.896338 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 16:54:41.896338 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 16:54:41.896338 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:41.896338 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:41.896338 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:41.896338 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:41.896338 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:41.896338 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:41.897162 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:41.897162 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:41.897162 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:41.897162 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:41.897162 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:41.897162 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:41.897162 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:41.897162 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:41.897162 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:41.897162 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:41.897162 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 16:54:41.897162 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 16:54:41.897162 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 18 16:54:41.897162 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:41.899804 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 347/714] 10.222.0.82 () {56 vars in 1129 bytes} [Thu Aug 18 16:54:41 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:41.900818 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8a1cd5-7189-42fa-9fbe-c114dc252d49 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:41.951988 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff45526-f9ad-4dde-841f-7d88d34cb521 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:42.004885 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a482b9c-2546-44e7-bdd2-870eaddc01d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:42.005741 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a482b9c-2546-44e7-bdd2-870eaddc01d5 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:42.006608 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a482b9c-2546-44e7-bdd2-870eaddc01d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:42.007083 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a482b9c-2546-44e7-bdd2-870eaddc01d5 None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:42.008261 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a482b9c-2546-44e7-bdd2-870eaddc01d5 None admin] RBAC: Adding query filter params (name=network) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:42.008725 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a482b9c-2546-44e7-bdd2-870eaddc01d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:42.010781 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a482b9c-2546-44e7-bdd2-870eaddc01d5 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:42.023028 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 368/715] 10.222.0.82 () {56 vars in 1112 bytes} [Thu Aug 18 16:54:41 2022] GET /identity/v3/services?name=network => generated 124 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:42.024348 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a482b9c-2546-44e7-bdd2-870eaddc01d5 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:42.037626 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8a1cd5-7189-42fa-9fbe-c114dc252d49 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:42.055728 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:42.080198 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a7e617-05e2-4398-bba8-7f263a27d19e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:42.081316 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a7e617-05e2-4398-bba8-7f263a27d19e None admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:42.081706 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a7e617-05e2-4398-bba8-7f263a27d19e None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:42.082082 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a7e617-05e2-4398-bba8-7f263a27d19e None admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:42.083154 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a7e617-05e2-4398-bba8-7f263a27d19e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:42.084968 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a7e617-05e2-4398-bba8-7f263a27d19e None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:42.119334 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:42.144439 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a482b9c-2546-44e7-bdd2-870eaddc01d5 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:42.178183 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b15878-6e7a-48b1-b9ac-370629579e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:42.178924 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b15878-6e7a-48b1-b9ac-370629579e8e None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:42.179212 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b15878-6e7a-48b1-b9ac-370629579e8e None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:42.179478 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b15878-6e7a-48b1-b9ac-370629579e8e None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:42.180125 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b15878-6e7a-48b1-b9ac-370629579e8e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:42.180423 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b15878-6e7a-48b1-b9ac-370629579e8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:42.181784 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b15878-6e7a-48b1-b9ac-370629579e8e None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:42.190940 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 369/716] 10.222.0.82 () {56 vars in 1124 bytes} [Thu Aug 18 16:54:42 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:42.190940 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b15878-6e7a-48b1-b9ac-370629579e8e None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:42.198671 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 348/717] 10.222.0.82 () {60 vars in 1132 bytes} [Thu Aug 18 16:54:41 2022] POST /identity/v3/users => generated 254 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 16:54:42.198671 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a7e617-05e2-4398-bba8-7f263a27d19e None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:42.333818 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a7e617-05e2-4398-bba8-7f263a27d19e None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:42.378225 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8cd1c8-103c-4995-a8b8-634fd1fc0031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:42.379043 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8cd1c8-103c-4995-a8b8-634fd1fc0031 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:42.379473 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8cd1c8-103c-4995-a8b8-634fd1fc0031 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:42.379848 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8cd1c8-103c-4995-a8b8-634fd1fc0031 None admin] PATH_INFO: `/v3/services` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:42.380731 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8cd1c8-103c-4995-a8b8-634fd1fc0031 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:42.381201 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8cd1c8-103c-4995-a8b8-634fd1fc0031 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:42.382845 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8cd1c8-103c-4995-a8b8-634fd1fc0031 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:42.389865 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 349/718] 10.222.0.82 () {56 vars in 1124 bytes} [Thu Aug 18 16:54:42 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:42.397230 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b15878-6e7a-48b1-b9ac-370629579e8e None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:42.462642 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e6c5df-fef6-4430-952b-eab6070d2c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:42.463543 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e6c5df-fef6-4430-952b-eab6070d2c25 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:42.463965 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e6c5df-fef6-4430-952b-eab6070d2c25 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:42.464381 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e6c5df-fef6-4430-952b-eab6070d2c25 None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:42.466158 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e6c5df-fef6-4430-952b-eab6070d2c25 None admin] RBAC: Adding query filter params (type=network) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:42.466914 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e6c5df-fef6-4430-952b-eab6070d2c25 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:42.468472 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e6c5df-fef6-4430-952b-eab6070d2c25 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:42.475413 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 370/719] 10.222.0.82 () {56 vars in 1112 bytes} [Thu Aug 18 16:54:42 2022] GET /identity/v3/services?type=network => generated 124 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:42.778185 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ea82e-fa7b-4731-9eef-5d7a84a28013 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:42.778483 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ea82e-fa7b-4731-9eef-5d7a84a28013 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:42.778761 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ea82e-fa7b-4731-9eef-5d7a84a28013 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:42.780709 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 350/720] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:54:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:54:42.786181 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7acdf6-8f6d-4e97-b309-d6aa17ed2b3d None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:42.786472 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7acdf6-8f6d-4e97-b309-d6aa17ed2b3d None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:42.786754 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7acdf6-8f6d-4e97-b309-d6aa17ed2b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:42.793693 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:42.807878 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-fd7acdf6-8f6d-4e97-b309-d6aa17ed2b3d None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:54:42.810011 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-fd7acdf6-8f6d-4e97-b309-d6aa17ed2b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:54:42.911829 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 371/721] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:54:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:54:43.265023 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4ea82e-fa7b-4731-9eef-5d7a84a28013 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:43.306089 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:43.387524 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4ea82e-fa7b-4731-9eef-5d7a84a28013 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:43.429913 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b2f545-76e5-45ac-9e29-c34339e6e687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:43.430524 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b2f545-76e5-45ac-9e29-c34339e6e687 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:43.430813 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b2f545-76e5-45ac-9e29-c34339e6e687 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:43.431117 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b2f545-76e5-45ac-9e29-c34339e6e687 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:43.450266 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b2f545-76e5-45ac-9e29-c34339e6e687 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:54:43.455679 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-46b2f545-76e5-45ac-9e29-c34339e6e687 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 16:54:43.455679 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:43.455679 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:43.455679 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:43.455679 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:43.455679 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:43.455679 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:43.455679 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:43.455679 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:43.455679 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:43.455679 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:43.455679 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:43.455679 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:54:43.455679 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:54:43.455679 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:54:43.455679 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:54:43.456608 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:54:43.456608 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:54:43.456608 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:54:43.456608 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:54:43.456608 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:43.456608 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:43.456608 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:43.456608 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:43.456608 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:43.456608 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:43.456608 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:43.456608 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:43.456608 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:43.456608 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:43.456608 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:43.456608 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:43.457422 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:43.457422 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:43.457422 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:43.457422 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:43.457422 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:54:43.457422 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:54:43.457422 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:54:43.457422 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:54:43.457422 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:54:43.457422 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:54:43.457422 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 16:54:43.457422 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:43.458044 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 351/722] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 16:54:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:43.462535 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7acdf6-8f6d-4e97-b309-d6aa17ed2b3d None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:43.598172 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792892b1-fd0a-4d52-9ca3-e17076044f66 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:43.598546 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792892b1-fd0a-4d52-9ca3-e17076044f66 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:43.598724 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792892b1-fd0a-4d52-9ca3-e17076044f66 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:43.601087 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 352/723] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:54:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:54:43.604858 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d56b111-f11f-4436-8f41-a866728d80d4 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:43.605150 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d56b111-f11f-4436-8f41-a866728d80d4 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:43.605431 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d56b111-f11f-4436-8f41-a866728d80d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:43.615652 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7acdf6-8f6d-4e97-b309-d6aa17ed2b3d None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:43.647260 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:43.648410 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-7d56b111-f11f-4436-8f41-a866728d80d4 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:54:43.649730 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-7d56b111-f11f-4436-8f41-a866728d80d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:54:43.662558 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c240f8-5ef2-4f59-be7e-489947ebbf07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:43.663494 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c240f8-5ef2-4f59-be7e-489947ebbf07 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:43.666759 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c240f8-5ef2-4f59-be7e-489947ebbf07 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:43.666759 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c240f8-5ef2-4f59-be7e-489947ebbf07 None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:43.666759 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c240f8-5ef2-4f59-be7e-489947ebbf07 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:43.666759 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c240f8-5ef2-4f59-be7e-489947ebbf07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:43.666759 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c240f8-5ef2-4f59-be7e-489947ebbf07 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:43.683012 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 372/724] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:54:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:43.693111 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c240f8-5ef2-4f59-be7e-489947ebbf07 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:43.760366 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 353/725] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:54:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:54:43.833776 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c240f8-5ef2-4f59-be7e-489947ebbf07 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:43.891723 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029a5b83-a03c-475a-8a4b-0212b9da0b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:43.892304 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a5b83-a03c-475a-8a4b-0212b9da0b58 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:43.892592 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a5b83-a03c-475a-8a4b-0212b9da0b58 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:43.892930 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a5b83-a03c-475a-8a4b-0212b9da0b58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:43.906815 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029a5b83-a03c-475a-8a4b-0212b9da0b58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:43.908392 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029a5b83-a03c-475a-8a4b-0212b9da0b58 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:43.924570 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-029a5b83-a03c-475a-8a4b-0212b9da0b58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:43.924570 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:43.924570 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:43.924570 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:43.924570 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:43.924570 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:43.924570 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:43.924570 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:43.924570 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:43.924570 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:43.924570 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:43.924570 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:43.924570 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:54:43.924570 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:54:43.924570 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:54:43.924570 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:54:43.925522 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:43.925522 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:43.925522 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:43.925522 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:43.925522 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:43.925522 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:43.925522 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:43.925522 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:43.925522 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:43.925522 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:43.925522 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:43.925522 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:43.925522 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:43.925522 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:43.925522 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:43.925522 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:43.925522 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:54:43.926375 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:54:43.926375 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:43.926375 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:43.932365 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 373/726] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:54:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:43.938821 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d56b111-f11f-4436-8f41-a866728d80d4 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:43.993913 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:44.086267 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d56b111-f11f-4436-8f41-a866728d80d4 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:44.125945 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d8234e-24e6-4c45-8d09-6fe742c46d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:44.126562 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d8234e-24e6-4c45-8d09-6fe742c46d2d None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:44.126843 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d8234e-24e6-4c45-8d09-6fe742c46d2d None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:44.127138 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d8234e-24e6-4c45-8d09-6fe742c46d2d None admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:44.127777 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d8234e-24e6-4c45-8d09-6fe742c46d2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:44.128079 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d8234e-24e6-4c45-8d09-6fe742c46d2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:44.129474 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d8234e-24e6-4c45-8d09-6fe742c46d2d None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:44.142112 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 354/727] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:54:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:44.147520 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029a5b83-a03c-475a-8a4b-0212b9da0b58 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:44.176278 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d8234e-24e6-4c45-8d09-6fe742c46d2d None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:44.222923 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:44.295293 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:44.303218 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d8234e-24e6-4c45-8d09-6fe742c46d2d None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:44.333055 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029a5b83-a03c-475a-8a4b-0212b9da0b58 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:44.375609 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006a4484-5832-49f8-9c79-71cf444627cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:44.376263 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a4484-5832-49f8-9c79-71cf444627cf None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:44.376549 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a4484-5832-49f8-9c79-71cf444627cf None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:44.376825 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a4484-5832-49f8-9c79-71cf444627cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:44.398930 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006a4484-5832-49f8-9c79-71cf444627cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:54:44.404845 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-006a4484-5832-49f8-9c79-71cf444627cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 16:54:44.404845 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:44.404845 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:44.404845 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:44.404845 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:44.404845 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:44.404845 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:44.404845 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:44.404845 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:44.404845 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:44.404845 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:44.404845 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:44.404845 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:54:44.404845 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:54:44.404845 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:54:44.404845 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:54:44.405743 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:54:44.405743 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:54:44.405743 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:54:44.405743 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:54:44.405743 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:44.405743 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:44.405743 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:44.405743 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:44.405743 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:44.405743 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:44.405743 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:44.405743 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:44.405743 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:44.405743 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:44.405743 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:44.405743 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:44.406541 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:44.406541 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:44.406541 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:44.406541 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:44.406541 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:54:44.406541 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:54:44.406541 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:54:44.406541 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:54:44.406541 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:54:44.406541 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:54:44.406541 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 16:54:44.406541 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:44.406541 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 355/728] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 16:54:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 16:54:44.414567 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79efc922-2a7a-4113-90a0-6050e6076d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:44.415248 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79efc922-2a7a-4113-90a0-6050e6076d7c None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:44.415534 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79efc922-2a7a-4113-90a0-6050e6076d7c None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:44.415813 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79efc922-2a7a-4113-90a0-6050e6076d7c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:44.425086 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79efc922-2a7a-4113-90a0-6050e6076d7c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:44.426218 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006a4484-5832-49f8-9c79-71cf444627cf None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:44.426540 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79efc922-2a7a-4113-90a0-6050e6076d7c None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:44.451698 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-79efc922-2a7a-4113-90a0-6050e6076d7c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 18 16:54:44.451698 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:44.451698 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:44.451698 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:44.451698 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:44.451698 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:44.451698 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:44.451698 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:44.451698 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:44.451698 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:44.451698 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:44.451698 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:44.451698 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 16:54:44.451698 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 16:54:44.451698 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 16:54:44.451698 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 16:54:44.452667 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:44.452667 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:44.452667 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 16:54:44.452667 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:54:44.452667 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 16:54:44.452667 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:54:44.452667 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:44.452667 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:44.452667 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:44.452667 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:44.452667 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:44.452667 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:44.452667 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:44.452667 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:44.452667 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:44.452667 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:44.452667 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:44.452667 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:44.453565 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:44.453565 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:44.453565 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 16:54:44.453565 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 16:54:44.453565 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 16:54:44.453565 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 16:54:44.453565 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 16:54:44.453565 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 16:54:44.453565 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 18 16:54:44.453565 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:44.458408 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 374/729] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 16:54:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:44.463628 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79efc922-2a7a-4113-90a0-6050e6076d7c None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:44.613430 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006a4484-5832-49f8-9c79-71cf444627cf None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:44.649695 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79efc922-2a7a-4113-90a0-6050e6076d7c None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:44.659423 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:44.680252 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61b71aa-67f6-4b75-b548-94262378d14a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:44.680863 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b71aa-67f6-4b75-b548-94262378d14a None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:44.681142 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b71aa-67f6-4b75-b548-94262378d14a None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:44.681473 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b71aa-67f6-4b75-b548-94262378d14a None admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:44.682137 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61b71aa-67f6-4b75-b548-94262378d14a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:44.682487 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61b71aa-67f6-4b75-b548-94262378d14a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:44.684614 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61b71aa-67f6-4b75-b548-94262378d14a None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:44.694762 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 356/730] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:54:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:44.701072 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61b71aa-67f6-4b75-b548-94262378d14a None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:44.701232 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46498f0-d53b-4dbd-b265-a98d3c6adc41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:44.701863 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46498f0-d53b-4dbd-b265-a98d3c6adc41 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:44.702324 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46498f0-d53b-4dbd-b265-a98d3c6adc41 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:44.702607 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46498f0-d53b-4dbd-b265-a98d3c6adc41 None admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:44.703381 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46498f0-d53b-4dbd-b265-a98d3c6adc41 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:44.703883 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46498f0-d53b-4dbd-b265-a98d3c6adc41 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:44.706289 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46498f0-d53b-4dbd-b265-a98d3c6adc41 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:44.710057 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:44.727928 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 375/731] 10.222.0.82 () {56 vars in 1135 bytes} [Thu Aug 18 16:54:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:44.733636 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46498f0-d53b-4dbd-b265-a98d3c6adc41 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:44.834788 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61b71aa-67f6-4b75-b548-94262378d14a None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:44.846936 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46498f0-d53b-4dbd-b265-a98d3c6adc41 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:44.873075 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9903690-5815-4512-86aa-8900c40f4bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:44.873805 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9903690-5815-4512-86aa-8900c40f4bf3 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:44.874085 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9903690-5815-4512-86aa-8900c40f4bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:44.874359 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9903690-5815-4512-86aa-8900c40f4bf3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:44.884037 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5840cf5b-62f2-4296-85ed-7828eee96b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:44.884647 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840cf5b-62f2-4296-85ed-7828eee96b89 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:44.884929 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840cf5b-62f2-4296-85ed-7828eee96b89 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:44.885047 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9903690-5815-4512-86aa-8900c40f4bf3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:44.885228 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840cf5b-62f2-4296-85ed-7828eee96b89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:44.886399 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9903690-5815-4512-86aa-8900c40f4bf3 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:44.901261 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-c9903690-5815-4512-86aa-8900c40f4bf3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:44.901261 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:44.901261 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:44.901261 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:44.901261 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:44.901261 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:44.901261 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:44.901261 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:44.901261 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:44.901261 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:44.901261 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:44.901261 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:44.901261 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:54:44.901261 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:54:44.901261 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:54:44.901261 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:54:44.902486 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:44.902486 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:44.902486 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:44.902486 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:44.902486 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:44.902486 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:44.902486 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:44.902486 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:44.902486 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:44.902486 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:44.902486 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:44.902486 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:44.902486 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:44.902486 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:44.902486 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:44.902486 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:44.902486 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:54:44.904074 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:54:44.904074 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:44.904074 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:44.904074 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 357/732] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:54:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 16:54:44.904528 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795aae5f-5231-4e23-99e5-57bf9d904df3 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:44.904811 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795aae5f-5231-4e23-99e5-57bf9d904df3 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:44.904971 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795aae5f-5231-4e23-99e5-57bf9d904df3 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:44.906331 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 358/733] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:54:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:54:44.911244 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b21751-7a9b-4178-801e-2358c1edbfe9 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:44.911706 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b21751-7a9b-4178-801e-2358c1edbfe9 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:44.911835 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b21751-7a9b-4178-801e-2358c1edbfe9 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:44.914946 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 359/734] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:54:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:54:44.914946 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b21751-7a9b-4178-801e-2358c1edbfe9 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:44.929947 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:44.970922 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5840cf5b-62f2-4296-85ed-7828eee96b89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:44.972630 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5840cf5b-62f2-4296-85ed-7828eee96b89 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:44.976128 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:45.020498 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b21751-7a9b-4178-801e-2358c1edbfe9 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:45.053881 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efecd1cb-4732-40d6-8a51-7594af37d64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:45.054492 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efecd1cb-4732-40d6-8a51-7594af37d64e None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:45.054772 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efecd1cb-4732-40d6-8a51-7594af37d64e None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:45.055086 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efecd1cb-4732-40d6-8a51-7594af37d64e None admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:45.055718 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efecd1cb-4732-40d6-8a51-7594af37d64e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:45.056027 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efecd1cb-4732-40d6-8a51-7594af37d64e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:45.057406 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efecd1cb-4732-40d6-8a51-7594af37d64e None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:45.069782 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 360/735] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:54:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:45.070953 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e493994-335e-4c82-9890-88a7ad2b862d None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:45.071239 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e493994-335e-4c82-9890-88a7ad2b862d None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:45.071510 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e493994-335e-4c82-9890-88a7ad2b862d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:45.078993 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 376/736] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:54:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 18 16:54:45.078993 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b9a1a-18c7-4a2b-a687-65d3894a84a3 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:45.078993 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b9a1a-18c7-4a2b-a687-65d3894a84a3 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:45.078993 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b9a1a-18c7-4a2b-a687-65d3894a84a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:45.090547 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-6e493994-335e-4c82-9890-88a7ad2b862d None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:54:45.091772 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-6e493994-335e-4c82-9890-88a7ad2b862d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:54:45.095887 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-694b9a1a-18c7-4a2b-a687-65d3894a84a3 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:54:45.097065 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-694b9a1a-18c7-4a2b-a687-65d3894a84a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:54:45.155284 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:45.157736 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 361/737] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:54:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:54:45.158847 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7ddea-871c-4c63-9233-4598d7230f31 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:45.159249 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7ddea-871c-4c63-9233-4598d7230f31 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:45.159603 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7ddea-871c-4c63-9233-4598d7230f31 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:45.161004 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 362/738] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:54:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:54:45.172695 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee7ddea-871c-4c63-9233-4598d7230f31 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:45.172843 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 377/739] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:54:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:54:45.172843 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694b9a1a-18c7-4a2b-a687-65d3894a84a3 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:45.245131 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:45.282222 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694b9a1a-18c7-4a2b-a687-65d3894a84a3 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:45.312148 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee7ddea-871c-4c63-9233-4598d7230f31 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:45.325864 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce639b1-be36-49ab-837d-5a1e2a0520b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:45.326816 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce639b1-be36-49ab-837d-5a1e2a0520b0 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:45.327319 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce639b1-be36-49ab-837d-5a1e2a0520b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:45.327887 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce639b1-be36-49ab-837d-5a1e2a0520b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:45.340783 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce639b1-be36-49ab-837d-5a1e2a0520b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:45.342886 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce639b1-be36-49ab-837d-5a1e2a0520b0 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:45.351204 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d346e6-f5c2-46c3-bbcd-8c9f30a0723c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:45.351891 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d346e6-f5c2-46c3-bbcd-8c9f30a0723c None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:45.355796 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d346e6-f5c2-46c3-bbcd-8c9f30a0723c None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:45.355796 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d346e6-f5c2-46c3-bbcd-8c9f30a0723c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:45.360090 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-9ce639b1-be36-49ab-837d-5a1e2a0520b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:45.360090 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:45.360090 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:45.360090 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:45.360090 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:45.360090 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:45.360090 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:45.360090 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:45.360090 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:45.360090 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:45.360090 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:45.360090 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:45.360090 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:54:45.360090 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:54:45.360090 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:54:45.360090 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:54:45.361216 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:45.361216 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:45.361216 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:45.361216 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:45.361216 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:45.361216 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:45.361216 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:45.361216 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:45.361216 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:45.361216 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:45.361216 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:45.361216 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:45.361216 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:45.361216 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:45.361216 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:45.361216 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:45.361216 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:54:45.362307 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:54:45.362307 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:45.362307 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:45.362507 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d346e6-f5c2-46c3-bbcd-8c9f30a0723c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:45.363512 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d346e6-f5c2-46c3-bbcd-8c9f30a0723c None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:45.364213 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 378/740] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:54:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:45.365514 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ece201-64d9-488f-aed3-090fff3fab23 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:45.365932 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ece201-64d9-488f-aed3-090fff3fab23 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:45.366338 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ece201-64d9-488f-aed3-090fff3fab23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:45.381911 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-33d346e6-f5c2-46c3-bbcd-8c9f30a0723c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 18 16:54:45.381911 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:45.381911 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:45.381911 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:45.381911 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:45.381911 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:45.381911 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:45.381911 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:45.381911 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:45.381911 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:45.381911 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:45.381911 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:45.381911 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 16:54:45.381911 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 16:54:45.381911 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 16:54:45.381911 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 16:54:45.383095 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:45.383095 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:45.383095 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 16:54:45.383095 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:54:45.383095 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 16:54:45.383095 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:54:45.383095 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:45.383095 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:45.383095 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:45.383095 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:45.383095 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:45.383095 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:45.383095 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:45.383095 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:45.383095 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:45.383095 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:45.383095 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:45.383095 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:45.384022 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:45.384022 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:45.384022 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 16:54:45.384022 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 16:54:45.384022 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 16:54:45.384022 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 16:54:45.384022 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 16:54:45.384022 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 16:54:45.384022 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 18 16:54:45.384022 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:45.388777 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 363/741] 10.222.0.82 () {56 vars in 1093 bytes} [Thu Aug 18 16:54:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:45.389439 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-a3ece201-64d9-488f-aed3-090fff3fab23 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:54:45.389602 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d346e6-f5c2-46c3-bbcd-8c9f30a0723c None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:45.391031 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-a3ece201-64d9-488f-aed3-090fff3fab23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:54:45.442639 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:45.468367 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 379/742] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:54:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 16:54:45.469204 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ece201-64d9-488f-aed3-090fff3fab23 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:45.506367 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d346e6-f5c2-46c3-bbcd-8c9f30a0723c None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:45.517443 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:45.542728 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e903fc93-c8e7-49f2-8d1e-05b23252dd2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:45.544084 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903fc93-c8e7-49f2-8d1e-05b23252dd2d None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:45.544366 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903fc93-c8e7-49f2-8d1e-05b23252dd2d None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:45.544635 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903fc93-c8e7-49f2-8d1e-05b23252dd2d None admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:45.545264 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e903fc93-c8e7-49f2-8d1e-05b23252dd2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:45.545563 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e903fc93-c8e7-49f2-8d1e-05b23252dd2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:45.546988 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e903fc93-c8e7-49f2-8d1e-05b23252dd2d None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:45.560093 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 364/743] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:54:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:45.560783 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e903fc93-c8e7-49f2-8d1e-05b23252dd2d None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:45.578149 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ece201-64d9-488f-aed3-090fff3fab23 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:45.614760 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228f68b7-3fa1-4e16-b449-806f36e07561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:45.615467 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228f68b7-3fa1-4e16-b449-806f36e07561 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:45.615676 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228f68b7-3fa1-4e16-b449-806f36e07561 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:45.615955 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228f68b7-3fa1-4e16-b449-806f36e07561 None admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:45.616678 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228f68b7-3fa1-4e16-b449-806f36e07561 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:45.616978 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228f68b7-3fa1-4e16-b449-806f36e07561 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:45.618402 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228f68b7-3fa1-4e16-b449-806f36e07561 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:45.635903 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 380/744] 10.222.0.82 () {56 vars in 1133 bytes} [Thu Aug 18 16:54:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:45.639914 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228f68b7-3fa1-4e16-b449-806f36e07561 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:45.667043 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e903fc93-c8e7-49f2-8d1e-05b23252dd2d None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:45.683273 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:45.703538 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15e0622-4ecd-437a-93a4-9a5d8315d09c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:45.703538 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15e0622-4ecd-437a-93a4-9a5d8315d09c None admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:45.703538 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15e0622-4ecd-437a-93a4-9a5d8315d09c None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:45.703538 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15e0622-4ecd-437a-93a4-9a5d8315d09c None admin] PATH_INFO: `/v3/services` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:45.703947 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15e0622-4ecd-437a-93a4-9a5d8315d09c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:45.705335 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15e0622-4ecd-437a-93a4-9a5d8315d09c None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:45.751117 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228f68b7-3fa1-4e16-b449-806f36e07561 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:45.784040 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a3872-36e6-43fb-9599-8b9eaaf5da96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:45.784670 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a3872-36e6-43fb-9599-8b9eaaf5da96 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:45.784948 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a3872-36e6-43fb-9599-8b9eaaf5da96 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:45.785214 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a3872-36e6-43fb-9599-8b9eaaf5da96 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:45.795573 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520a3872-36e6-43fb-9599-8b9eaaf5da96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:54:45.800278 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-520a3872-36e6-43fb-9599-8b9eaaf5da96 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 16:54:45.800278 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:45.800278 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:45.800278 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:45.800278 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:45.800278 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:45.800278 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:45.800278 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:45.800278 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:45.800278 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:45.800278 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:45.800278 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:45.800278 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:54:45.800278 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:54:45.800278 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:54:45.800278 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:54:45.801418 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:54:45.801418 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:54:45.801418 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:54:45.801418 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:54:45.801418 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:45.801418 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:45.801418 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:45.801418 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:45.801418 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:45.801418 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:45.801418 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:45.801418 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:45.801418 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:45.801418 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:45.801418 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:45.801418 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:45.802269 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:45.802269 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:45.802269 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:45.802269 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:45.802269 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:54:45.802269 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:54:45.802269 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:54:45.802269 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:54:45.802269 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:54:45.802269 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:54:45.802269 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 16:54:45.802269 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:45.802269 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 381/745] 10.222.0.82 () {56 vars in 1102 bytes} [Thu Aug 18 16:54:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:45.802269 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a3872-36e6-43fb-9599-8b9eaaf5da96 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:45.818984 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 365/746] 10.222.0.82 () {60 vars in 1141 bytes} [Thu Aug 18 16:54:45 2022] POST /identity/v3/services => generated 251 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 16:54:45.818984 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15e0622-4ecd-437a-93a4-9a5d8315d09c None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:45.849109 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:45.912629 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:45.921633 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a3872-36e6-43fb-9599-8b9eaaf5da96 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:45.943396 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15e0622-4ecd-437a-93a4-9a5d8315d09c None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:45.960630 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8244bfeb-3845-42cb-b33b-c1f097389aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:45.961488 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8244bfeb-3845-42cb-b33b-c1f097389aad None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:45.961920 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8244bfeb-3845-42cb-b33b-c1f097389aad None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:45.962191 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8244bfeb-3845-42cb-b33b-c1f097389aad None admin] PATH_INFO: `/v3/projects/service` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:45.969849 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8244bfeb-3845-42cb-b33b-c1f097389aad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:45.971533 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8244bfeb-3845-42cb-b33b-c1f097389aad None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:45.983235 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-8244bfeb-3845-42cb-b33b-c1f097389aad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:54:45.983235 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:45.983235 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:45.983235 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:45.983235 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:45.983235 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:45.983235 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:45.983235 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:45.983235 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:45.983235 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:45.983235 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:45.983235 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:45.983235 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 16:54:45.983235 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 16:54:45.983235 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 16:54:45.984114 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 16:54:45.984114 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:45.984114 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:45.984114 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:45.984114 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:45.984114 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:45.984114 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:45.984114 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:45.984114 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:45.984114 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:45.984114 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:45.984114 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:45.984114 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:45.984114 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:45.984114 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:45.984114 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:45.984114 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:45.985142 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 16:54:45.985142 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 16:54:45.985142 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 16:54:45.985142 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 16:54:45.985142 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 16:54:45.985142 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 16:54:45.985142 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:54:45.985142 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:45.985142 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 382/747] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 16:54:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:45.985887 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8244bfeb-3845-42cb-b33b-c1f097389aad None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:45.992488 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee0b122-fce4-4aa7-a198-1c8e32928794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:45.993370 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee0b122-fce4-4aa7-a198-1c8e32928794 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:45.993660 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee0b122-fce4-4aa7-a198-1c8e32928794 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:45.993932 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee0b122-fce4-4aa7-a198-1c8e32928794 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:46.005526 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee0b122-fce4-4aa7-a198-1c8e32928794 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:46.007017 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee0b122-fce4-4aa7-a198-1c8e32928794 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:46.022949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-aee0b122-fce4-4aa7-a198-1c8e32928794 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:46.022949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:46.022949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:46.022949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:46.022949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:46.022949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:46.022949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:46.022949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:46.022949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:46.022949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:46.022949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:46.022949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:46.022949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:54:46.022949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:54:46.022949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:54:46.022949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:54:46.023925 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:46.023925 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:46.023925 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:46.023925 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:46.023925 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:46.023925 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:46.023925 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:46.023925 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:46.023925 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:46.023925 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:46.023925 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:46.023925 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:46.023925 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:46.023925 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:46.023925 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:46.023925 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:46.023925 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:54:46.024781 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:54:46.024781 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:46.024781 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:46.024781 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 366/748] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:54:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:46.024998 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee0b122-fce4-4aa7-a198-1c8e32928794 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:46.091303 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8244bfeb-3845-42cb-b33b-c1f097389aad None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:46.112846 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:46.126392 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a720e4-f79c-4e85-beea-91dba2d640da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:46.127041 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a720e4-f79c-4e85-beea-91dba2d640da None admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:46.127329 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a720e4-f79c-4e85-beea-91dba2d640da None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:46.127611 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a720e4-f79c-4e85-beea-91dba2d640da None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:46.128267 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a720e4-f79c-4e85-beea-91dba2d640da None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:46.129635 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a720e4-f79c-4e85-beea-91dba2d640da None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:46.137405 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee0b122-fce4-4aa7-a198-1c8e32928794 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:46.171508 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ea575f-72fa-45c2-89e1-e7de1440b14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:46.172116 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea575f-72fa-45c2-89e1-e7de1440b14c None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:46.172394 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea575f-72fa-45c2-89e1-e7de1440b14c None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:46.172665 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea575f-72fa-45c2-89e1-e7de1440b14c None admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:46.173317 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ea575f-72fa-45c2-89e1-e7de1440b14c None admin] RBAC: Adding query filter params (name=service) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:46.173632 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ea575f-72fa-45c2-89e1-e7de1440b14c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:46.175005 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ea575f-72fa-45c2-89e1-e7de1440b14c None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:46.183159 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 367/749] 10.222.0.82 () {56 vars in 1103 bytes} [Thu Aug 18 16:54:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:46.183873 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ea575f-72fa-45c2-89e1-e7de1440b14c None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:46.201688 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 383/750] 10.222.0.82 () {60 vars in 1141 bytes} [Thu Aug 18 16:54:45 2022] POST /identity/v3/services => generated 240 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 16:54:46.202758 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a720e4-f79c-4e85-beea-91dba2d640da None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:46.233776 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:46.320496 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ea575f-72fa-45c2-89e1-e7de1440b14c None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:46.367797 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a720e4-f79c-4e85-beea-91dba2d640da None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:46.372965 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee868c71-a86a-4676-af95-f1ba3c163b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:46.373597 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee868c71-a86a-4676-af95-f1ba3c163b66 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:46.373881 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee868c71-a86a-4676-af95-f1ba3c163b66 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:46.374154 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee868c71-a86a-4676-af95-f1ba3c163b66 None admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:46.374854 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee868c71-a86a-4676-af95-f1ba3c163b66 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:46.375178 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee868c71-a86a-4676-af95-f1ba3c163b66 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:46.376634 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee868c71-a86a-4676-af95-f1ba3c163b66 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:46.386592 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:46.391957 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 368/751] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 16:54:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:46.392788 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee868c71-a86a-4676-af95-f1ba3c163b66 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:46.409974 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a482b35-d927-4bf7-b571-c892f3d16393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:46.410592 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a482b35-d927-4bf7-b571-c892f3d16393 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:46.410870 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a482b35-d927-4bf7-b571-c892f3d16393 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:46.411193 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a482b35-d927-4bf7-b571-c892f3d16393 None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:46.413142 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a482b35-d927-4bf7-b571-c892f3d16393 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:46.413142 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a482b35-d927-4bf7-b571-c892f3d16393 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:46.413628 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a482b35-d927-4bf7-b571-c892f3d16393 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:46.428784 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 384/752] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:54:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:46.429949 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a482b35-d927-4bf7-b571-c892f3d16393 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:46.508570 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:46.540016 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee868c71-a86a-4676-af95-f1ba3c163b66 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:46.541566 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a482b35-d927-4bf7-b571-c892f3d16393 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:46.581364 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5bcae9-76cf-4fa2-a3ce-9d1265f06643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:46.582261 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5bcae9-76cf-4fa2-a3ce-9d1265f06643 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:46.582742 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5bcae9-76cf-4fa2-a3ce-9d1265f06643 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:46.583278 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5bcae9-76cf-4fa2-a3ce-9d1265f06643 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:46.584353 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5bcae9-76cf-4fa2-a3ce-9d1265f06643 None admin] RBAC: Adding query filter params (role.id=2c6966cfb10f4d72b0242e9fb5e13b17, scope.project.id=2293c0aab61f44849d0d48b214cde7ea, user.id=1381673f81584f4b8084823e3aa99e2a) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:46.584870 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5bcae9-76cf-4fa2-a3ce-9d1265f06643 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:46.586102 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831f7c03-1654-4ecc-be00-67c657ac2174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:46.586800 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831f7c03-1654-4ecc-be00-67c657ac2174 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:46.587054 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5bcae9-76cf-4fa2-a3ce-9d1265f06643 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:46.587169 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831f7c03-1654-4ecc-be00-67c657ac2174 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:46.587545 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831f7c03-1654-4ecc-be00-67c657ac2174 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:46.594285 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 385/753] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 16:54:46 2022] GET /identity/v3/role_assignments?user.id=1381673f81584f4b8084823e3aa99e2a&scope.project.id=2293c0aab61f44849d0d48b214cde7ea&role.id=2c6966cfb10f4d72b0242e9fb5e13b17 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:46.595355 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5bcae9-76cf-4fa2-a3ce-9d1265f06643 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:46.612334 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831f7c03-1654-4ecc-be00-67c657ac2174 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:46.614161 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831f7c03-1654-4ecc-be00-67c657ac2174 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:46.645184 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-831f7c03-1654-4ecc-be00-67c657ac2174 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:46.645184 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:46.645184 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:46.645184 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:46.645184 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:46.645184 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:46.645184 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:46.645184 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:46.645184 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:46.645184 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:46.645184 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:46.645184 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:46.645184 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:54:46.645184 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:54:46.645184 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:54:46.645184 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:54:46.646185 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:46.646185 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:46.646185 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:46.646185 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:46.646185 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:46.646185 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:46.646185 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:46.646185 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:46.646185 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:46.646185 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:46.646185 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:46.646185 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:46.646185 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:46.646185 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:46.646185 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:46.646185 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:46.646185 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:54:46.647054 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:54:46.647054 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:46.647054 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:46.650019 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 369/754] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:54:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:46.663155 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831f7c03-1654-4ecc-be00-67c657ac2174 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:46.743162 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5bcae9-76cf-4fa2-a3ce-9d1265f06643 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:46.784018 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb83c54-e99f-45c6-8a0a-7fe24b99bd3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:46.784894 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb83c54-e99f-45c6-8a0a-7fe24b99bd3c None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:46.785245 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb83c54-e99f-45c6-8a0a-7fe24b99bd3c None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:46.785591 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb83c54-e99f-45c6-8a0a-7fe24b99bd3c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:46.787893 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:46.794088 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:46.795054 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb83c54-e99f-45c6-8a0a-7fe24b99bd3c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:46.805094 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb83c54-e99f-45c6-8a0a-7fe24b99bd3c None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:46.809839 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-1cb83c54-e99f-45c6-8a0a-7fe24b99bd3c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:54:46.809839 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:46.809839 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:46.809839 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:46.809839 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:46.809839 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:46.809839 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:46.809839 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:46.809839 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:46.809839 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:46.809839 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:46.809839 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:46.809839 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 16:54:46.809839 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 16:54:46.809839 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 16:54:46.810731 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 16:54:46.810731 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:46.810731 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:46.810731 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:46.810731 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:46.810731 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:46.810731 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:46.810731 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:46.810731 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:46.810731 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:46.810731 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:46.810731 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:46.810731 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:46.810731 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:46.810731 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:46.810731 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:46.810731 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:46.811843 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 16:54:46.811843 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 16:54:46.811843 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 16:54:46.811843 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 16:54:46.811843 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 16:54:46.811843 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 16:54:46.811843 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:54:46.811843 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:46.811843 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 386/755] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 16:54:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:46.817509 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb83c54-e99f-45c6-8a0a-7fe24b99bd3c None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:46.869919 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831f7c03-1654-4ecc-be00-67c657ac2174 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:46.906136 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe1b96b-ce7f-4aea-95e8-5403400e07fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:46.906846 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe1b96b-ce7f-4aea-95e8-5403400e07fb None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:46.912661 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe1b96b-ce7f-4aea-95e8-5403400e07fb None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:46.912996 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe1b96b-ce7f-4aea-95e8-5403400e07fb None admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:46.913783 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe1b96b-ce7f-4aea-95e8-5403400e07fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:46.914098 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe1b96b-ce7f-4aea-95e8-5403400e07fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:46.919576 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe1b96b-ce7f-4aea-95e8-5403400e07fb None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:46.941158 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb83c54-e99f-45c6-8a0a-7fe24b99bd3c None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:46.944154 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 370/756] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:54:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:46.949825 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe1b96b-ce7f-4aea-95e8-5403400e07fb None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:46.988211 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ee80ee-e177-4902-8c1c-c96fd41f9f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:46.988831 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee80ee-e177-4902-8c1c-c96fd41f9f45 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:46.989113 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee80ee-e177-4902-8c1c-c96fd41f9f45 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:46.989386 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee80ee-e177-4902-8c1c-c96fd41f9f45 None admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:46.990136 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ee80ee-e177-4902-8c1c-c96fd41f9f45 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:46.990443 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ee80ee-e177-4902-8c1c-c96fd41f9f45 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:46.991902 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ee80ee-e177-4902-8c1c-c96fd41f9f45 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:47.025558 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 387/757] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 16:54:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:47.028509 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ee80ee-e177-4902-8c1c-c96fd41f9f45 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:47.141529 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:47.188275 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ee80ee-e177-4902-8c1c-c96fd41f9f45 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:47.193727 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe1b96b-ce7f-4aea-95e8-5403400e07fb None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:47.221325 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:47.228509 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8279c36d-59ba-4b11-af89-e48cd1058360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:47.229179 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8279c36d-59ba-4b11-af89-e48cd1058360 None admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:47.229463 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8279c36d-59ba-4b11-af89-e48cd1058360 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:47.229881 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8279c36d-59ba-4b11-af89-e48cd1058360 None admin] PATH_INFO: `/v3/projects/2293c0aab61f44849d0d48b214cde7ea/users/55723c756cae4e7ab636054db48de02d/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:47.232112 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8279c36d-59ba-4b11-af89-e48cd1058360 None admin] RBAC: Authorizing `identity:create_grant(project_id=2293c0aab61f44849d0d48b214cde7ea, user_id=55723c756cae4e7ab636054db48de02d, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:47.233555 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8279c36d-59ba-4b11-af89-e48cd1058360 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:47.254308 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00010fd-be48-4532-9ee9-dd0a0dfc10b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:47.255007 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00010fd-be48-4532-9ee9-dd0a0dfc10b9 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:47.255295 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00010fd-be48-4532-9ee9-dd0a0dfc10b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:47.255571 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00010fd-be48-4532-9ee9-dd0a0dfc10b9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:47.263977 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00010fd-be48-4532-9ee9-dd0a0dfc10b9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:47.265508 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00010fd-be48-4532-9ee9-dd0a0dfc10b9 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:47.277942 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-d00010fd-be48-4532-9ee9-dd0a0dfc10b9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 18 16:54:47.277942 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:47.277942 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:47.277942 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:47.277942 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:47.277942 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:47.277942 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:47.277942 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:47.277942 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:47.277942 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:47.277942 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:47.277942 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:47.277942 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 16:54:47.277942 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 16:54:47.277942 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 16:54:47.277942 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 16:54:47.279289 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:47.279289 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:47.279289 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 16:54:47.279289 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:54:47.279289 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 16:54:47.279289 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:54:47.279289 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:47.279289 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:47.279289 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:47.279289 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:47.279289 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:47.279289 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:47.279289 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:47.279289 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:47.279289 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:47.279289 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:47.279289 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:47.279289 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:47.280371 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:47.280371 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:47.280371 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 16:54:47.280371 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 16:54:47.280371 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 16:54:47.280371 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 16:54:47.280371 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 16:54:47.280371 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 16:54:47.280371 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 18 16:54:47.280371 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:47.280371 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 371/758] 10.222.0.82 () {56 vars in 1117 bytes} [Thu Aug 18 16:54:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:47.288114 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00010fd-be48-4532-9ee9-dd0a0dfc10b9 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:47.321334 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 388/759] 10.222.0.82 () {58 vars in 1439 bytes} [Thu Aug 18 16:54:47 2022] PUT /identity/v3/projects/2293c0aab61f44849d0d48b214cde7ea/users/55723c756cae4e7ab636054db48de02d/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 16:54:47.406019 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00010fd-be48-4532-9ee9-dd0a0dfc10b9 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:47.444451 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aba469-e698-4bfd-a35d-9aa4e43c5621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:47.445102 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aba469-e698-4bfd-a35d-9aa4e43c5621 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:47.445386 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aba469-e698-4bfd-a35d-9aa4e43c5621 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:47.445660 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aba469-e698-4bfd-a35d-9aa4e43c5621 None admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:47.446438 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aba469-e698-4bfd-a35d-9aa4e43c5621 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:47.446745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aba469-e698-4bfd-a35d-9aa4e43c5621 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:47.448337 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aba469-e698-4bfd-a35d-9aa4e43c5621 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:47.469943 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 372/760] 10.222.0.82 () {56 vars in 1149 bytes} [Thu Aug 18 16:54:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:47.476214 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8279c36d-59ba-4b11-af89-e48cd1058360 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:47.607937 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8279c36d-59ba-4b11-af89-e48cd1058360 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:47.623773 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:47.646220 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52f7c16-3156-4c8d-86c1-310f97866ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:47.646832 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52f7c16-3156-4c8d-86c1-310f97866ff5 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:47.647142 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52f7c16-3156-4c8d-86c1-310f97866ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:47.647477 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52f7c16-3156-4c8d-86c1-310f97866ff5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:47.734303 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52f7c16-3156-4c8d-86c1-310f97866ff5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:47.735801 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52f7c16-3156-4c8d-86c1-310f97866ff5 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:47.875790 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb318e2a-46c7-4f34-bbd4-6094657e8bba None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:47.876315 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb318e2a-46c7-4f34-bbd4-6094657e8bba None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:47.876315 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb318e2a-46c7-4f34-bbd4-6094657e8bba None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:47.877880 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 373/761] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:54:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 16:54:47.882758 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba1efe-e137-4b2a-9ca2-a95e0b29f3ec None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:47.883073 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba1efe-e137-4b2a-9ca2-a95e0b29f3ec None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:47.883348 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba1efe-e137-4b2a-9ca2-a95e0b29f3ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:47.884317 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 389/762] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:54:47 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 16:54:47.890527 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52f7c16-3156-4c8d-86c1-310f97866ff5 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:47.904249 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-b7ba1efe-e137-4b2a-9ca2-a95e0b29f3ec None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:54:47.905873 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-b7ba1efe-e137-4b2a-9ca2-a95e0b29f3ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:54:47.916002 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:47.948043 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:48.012500 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 374/763] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:54:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 16:54:48.066443 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52f7c16-3156-4c8d-86c1-310f97866ff5 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:48.111617 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503965e8-c157-494f-874a-6d805f97e760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:48.112268 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503965e8-c157-494f-874a-6d805f97e760 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:48.112551 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503965e8-c157-494f-874a-6d805f97e760 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:48.112832 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503965e8-c157-494f-874a-6d805f97e760 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:48.123940 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503965e8-c157-494f-874a-6d805f97e760 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:48.125400 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503965e8-c157-494f-874a-6d805f97e760 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:48.146650 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-503965e8-c157-494f-874a-6d805f97e760 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:48.146650 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:48.146650 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:48.146650 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:48.146650 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:48.146650 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:48.146650 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:48.146650 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:48.146650 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:48.146650 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:48.146650 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:48.146650 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:48.146650 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:54:48.146650 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:54:48.146650 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:54:48.146650 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:54:48.147670 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:48.147670 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:48.147670 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:48.147670 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:48.147670 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:48.147670 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:48.147670 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:48.147670 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:48.147670 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:48.147670 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:48.147670 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:48.147670 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:48.147670 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:48.147670 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:48.147670 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:48.147670 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:48.147670 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:54:48.148537 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:54:48.148537 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:48.148537 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:48.152719 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 390/764] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:54:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 16:54:48.158478 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ba1efe-e137-4b2a-9ca2-a95e0b29f3ec None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:48.279331 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ba1efe-e137-4b2a-9ca2-a95e0b29f3ec None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:48.316313 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a00b2a9-9bc7-431d-a429-f5b74f8da7b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:48.316951 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a00b2a9-9bc7-431d-a429-f5b74f8da7b2 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:48.317230 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a00b2a9-9bc7-431d-a429-f5b74f8da7b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:48.317503 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a00b2a9-9bc7-431d-a429-f5b74f8da7b2 None admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:48.318162 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a00b2a9-9bc7-431d-a429-f5b74f8da7b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:48.318465 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a00b2a9-9bc7-431d-a429-f5b74f8da7b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:48.319893 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a00b2a9-9bc7-431d-a429-f5b74f8da7b2 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:48.336597 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 375/765] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:54:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:48.342930 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503965e8-c157-494f-874a-6d805f97e760 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:48.347021 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a00b2a9-9bc7-431d-a429-f5b74f8da7b2 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:48.375243 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:48.470126 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:48.524145 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503965e8-c157-494f-874a-6d805f97e760 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:48.526465 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a00b2a9-9bc7-431d-a429-f5b74f8da7b2 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:48.564488 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7316730f-0851-44fc-96fd-57cd759c3a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:48.565197 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7316730f-0851-44fc-96fd-57cd759c3a8f None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:48.565480 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7316730f-0851-44fc-96fd-57cd759c3a8f None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:48.565752 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7316730f-0851-44fc-96fd-57cd759c3a8f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:48.566327 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7316730f-0851-44fc-96fd-57cd759c3a8f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:48.567436 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd54b844-846e-48f9-afec-6f99284b6547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:48.567740 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7316730f-0851-44fc-96fd-57cd759c3a8f None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:48.568065 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd54b844-846e-48f9-afec-6f99284b6547 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:48.568529 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd54b844-846e-48f9-afec-6f99284b6547 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:48.568850 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd54b844-846e-48f9-afec-6f99284b6547 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:48.576718 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd54b844-846e-48f9-afec-6f99284b6547 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:48.577308 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-7316730f-0851-44fc-96fd-57cd759c3a8f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 18 16:54:48.577308 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:48.577308 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 16:54:48.577308 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 16:54:48.577308 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 16:54:48.577308 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 16:54:48.577308 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 16:54:48.577308 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 16:54:48.577308 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 18 16:54:48.577308 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:48.577308 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 16:54:48.577308 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:48.577308 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:48.577308 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:48.577308 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:48.577308 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:48.577308 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:48.578471 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:48.578471 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:48.578471 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:48.578471 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:48.578471 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:48.578471 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:48.578471 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 16:54:48.578471 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 16:54:48.578471 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 16:54:48.578471 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 16:54:48.578471 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:48.578471 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:48.578471 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:48.578471 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:48.578471 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:48.578471 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:48.579874 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:48.579874 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:48.579874 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:48.579874 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:48.579874 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:48.579874 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:48.579874 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:48.579874 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:48.579874 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:48.579874 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:48.579874 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 16:54:48.579874 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 16:54:48.579874 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 18 16:54:48.579874 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:48.581562 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd54b844-846e-48f9-afec-6f99284b6547 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:48.581721 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 376/766] 10.222.0.82 () {56 vars in 1129 bytes} [Thu Aug 18 16:54:48 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 16:54:48.581721 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522523f7-a00e-450b-b6a7-8ad8286dd51d None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:48.581721 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522523f7-a00e-450b-b6a7-8ad8286dd51d None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:48.581721 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522523f7-a00e-450b-b6a7-8ad8286dd51d None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:48.586047 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 377/767] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:54:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:54:48.586844 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522523f7-a00e-450b-b6a7-8ad8286dd51d None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:48.599337 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-bd54b844-846e-48f9-afec-6f99284b6547 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:54:48.599337 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:48.599337 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:48.599337 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:48.599337 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:48.599337 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:48.599337 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:48.599337 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:48.599337 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:48.599337 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:48.599337 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:48.599337 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:48.599337 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 16:54:48.599337 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 16:54:48.599337 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 16:54:48.601544 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 16:54:48.601544 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:48.601544 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:48.601544 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:48.601544 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:48.601544 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:48.601544 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:48.601544 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:48.601544 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:48.601544 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:48.601544 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:48.601544 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:48.601544 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:48.601544 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:48.601544 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:48.601544 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:48.601544 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:48.603804 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 16:54:48.603804 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 16:54:48.603804 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 16:54:48.603804 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 16:54:48.603804 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 16:54:48.603804 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 16:54:48.603804 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:54:48.603804 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:48.603804 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 391/768] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 16:54:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:48.603804 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14eb843-2ae3-437e-b97a-d9bdbfeb8506 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:48.603804 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14eb843-2ae3-437e-b97a-d9bdbfeb8506 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:48.605664 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14eb843-2ae3-437e-b97a-d9bdbfeb8506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:48.623171 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-e14eb843-2ae3-437e-b97a-d9bdbfeb8506 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:54:48.624511 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-e14eb843-2ae3-437e-b97a-d9bdbfeb8506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:54:48.696369 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:48.711755 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 392/769] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:54:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 16:54:48.712863 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14eb843-2ae3-437e-b97a-d9bdbfeb8506 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:48.725755 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522523f7-a00e-450b-b6a7-8ad8286dd51d None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:48.788799 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:48.813979 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef02e4e-e68a-4c49-bd2c-563869a8d55c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:48.814597 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef02e4e-e68a-4c49-bd2c-563869a8d55c None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:48.814875 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef02e4e-e68a-4c49-bd2c-563869a8d55c None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:48.815170 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef02e4e-e68a-4c49-bd2c-563869a8d55c None admin] PATH_INFO: `/v3/services` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:48.815798 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef02e4e-e68a-4c49-bd2c-563869a8d55c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:48.816098 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef02e4e-e68a-4c49-bd2c-563869a8d55c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:48.817544 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef02e4e-e68a-4c49-bd2c-563869a8d55c None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:48.831817 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 378/770] 10.222.0.82 () {56 vars in 1124 bytes} [Thu Aug 18 16:54:48 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:48.841512 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef02e4e-e68a-4c49-bd2c-563869a8d55c None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:48.881752 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14eb843-2ae3-437e-b97a-d9bdbfeb8506 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:48.941682 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a182c1-35f4-4bfe-a2ff-967039e52421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:48.942291 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a182c1-35f4-4bfe-a2ff-967039e52421 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:48.942569 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a182c1-35f4-4bfe-a2ff-967039e52421 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:48.942832 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a182c1-35f4-4bfe-a2ff-967039e52421 None admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:48.943547 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a182c1-35f4-4bfe-a2ff-967039e52421 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:48.943847 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a182c1-35f4-4bfe-a2ff-967039e52421 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:48.945284 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a182c1-35f4-4bfe-a2ff-967039e52421 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:48.973560 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 393/771] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 16:54:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:48.981256 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a182c1-35f4-4bfe-a2ff-967039e52421 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:49.042364 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef02e4e-e68a-4c49-bd2c-563869a8d55c None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:49.079994 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b43586-4870-4235-8e74-cb4e4b14fc1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:49.080620 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b43586-4870-4235-8e74-cb4e4b14fc1b None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:49.080906 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b43586-4870-4235-8e74-cb4e4b14fc1b None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:49.081186 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b43586-4870-4235-8e74-cb4e4b14fc1b None admin] PATH_INFO: `/v3/services` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:49.081818 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b43586-4870-4235-8e74-cb4e4b14fc1b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:49.082126 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b43586-4870-4235-8e74-cb4e4b14fc1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:49.083752 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b43586-4870-4235-8e74-cb4e4b14fc1b None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:49.089327 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 379/772] 10.222.0.82 () {56 vars in 1124 bytes} [Thu Aug 18 16:54:48 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:49.090683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b43586-4870-4235-8e74-cb4e4b14fc1b None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:49.110467 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:49.124619 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a182c1-35f4-4bfe-a2ff-967039e52421 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:49.142674 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:49.164066 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659c4ba5-ca51-4949-bdf9-875ca50024e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:49.165125 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659c4ba5-ca51-4949-bdf9-875ca50024e0 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:49.165669 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659c4ba5-ca51-4949-bdf9-875ca50024e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:49.166146 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659c4ba5-ca51-4949-bdf9-875ca50024e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:49.167315 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659c4ba5-ca51-4949-bdf9-875ca50024e0 None admin] RBAC: Adding query filter params (role.id=5e0f402e1af84b44a723ee0730315d74, scope.project.id=2293c0aab61f44849d0d48b214cde7ea, user.id=5de19a54ed104bcdb35ee4e5e793f1e7) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:49.170772 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659c4ba5-ca51-4949-bdf9-875ca50024e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:49.170772 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659c4ba5-ca51-4949-bdf9-875ca50024e0 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:49.177144 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 394/773] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 16:54:48 2022] GET /identity/v3/role_assignments?user.id=5de19a54ed104bcdb35ee4e5e793f1e7&scope.project.id=2293c0aab61f44849d0d48b214cde7ea&role.id=5e0f402e1af84b44a723ee0730315d74 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:49.178327 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659c4ba5-ca51-4949-bdf9-875ca50024e0 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:49.254828 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b43586-4870-4235-8e74-cb4e4b14fc1b None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:49.299648 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8253dec5-d010-4063-870e-2a7515a9724b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:49.300308 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8253dec5-d010-4063-870e-2a7515a9724b None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:49.300588 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8253dec5-d010-4063-870e-2a7515a9724b None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:49.300861 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8253dec5-d010-4063-870e-2a7515a9724b None admin] PATH_INFO: `/v3/services/network` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:49.301454 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8253dec5-d010-4063-870e-2a7515a9724b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:49.303044 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8253dec5-d010-4063-870e-2a7515a9724b None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:49.313745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-8253dec5-d010-4063-870e-2a7515a9724b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 18 16:54:49.313745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:49.313745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 16:54:49.313745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 16:54:49.313745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 16:54:49.313745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 16:54:49.313745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 16:54:49.313745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 16:54:49.313745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 18 16:54:49.313745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:49.313745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 16:54:49.313745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:49.313745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:49.313745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:49.313745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:49.313745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:49.313745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:49.314790 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:49.314790 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:49.314790 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:49.314790 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:49.314790 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:49.314790 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:49.314790 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 16:54:49.314790 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 16:54:49.314790 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 16:54:49.314790 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 16:54:49.314790 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:49.314790 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:49.314790 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:49.314790 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:49.314790 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:49.314790 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:49.316269 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:49.316269 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:49.316269 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:49.316269 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:49.316269 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:49.316269 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:49.316269 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:49.316269 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:49.316269 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:49.316269 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:49.316269 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 16:54:49.316269 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 16:54:49.316269 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 18 16:54:49.316269 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:49.316269 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 380/774] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 16:54:49 2022] GET /identity/v3/services/network => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:49.317378 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb49b36-2568-4f9a-9b5b-cb2961671dd6 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:49.317378 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb49b36-2568-4f9a-9b5b-cb2961671dd6 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:49.317378 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb49b36-2568-4f9a-9b5b-cb2961671dd6 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:49.322927 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 381/775] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:54:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:54:49.324569 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb49b36-2568-4f9a-9b5b-cb2961671dd6 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:49.427588 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:49.457451 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659c4ba5-ca51-4949-bdf9-875ca50024e0 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:49.462835 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb49b36-2568-4f9a-9b5b-cb2961671dd6 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:49.505112 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f04429d-f44f-49ba-b610-1530a3b2d070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:49.505757 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f04429d-f44f-49ba-b610-1530a3b2d070 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:49.506120 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f04429d-f44f-49ba-b610-1530a3b2d070 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:49.506399 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f04429d-f44f-49ba-b610-1530a3b2d070 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:49.507294 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f04429d-f44f-49ba-b610-1530a3b2d070 None admin] RBAC: Adding query filter params (interface=public, service_id=393d81dbb8af4b82a053e34bc06622c3, region_id=RegionOne) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:49.507690 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f04429d-f44f-49ba-b610-1530a3b2d070 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:49.509189 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfc06f4-53f9-4ee1-a0eb-824fc7e7c733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:49.509324 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f04429d-f44f-49ba-b610-1530a3b2d070 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:49.509914 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfc06f4-53f9-4ee1-a0eb-824fc7e7c733 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:49.510345 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfc06f4-53f9-4ee1-a0eb-824fc7e7c733 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:49.510688 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfc06f4-53f9-4ee1-a0eb-824fc7e7c733 None admin] PATH_INFO: `/v3/services` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:49.512277 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfc06f4-53f9-4ee1-a0eb-824fc7e7c733 None admin] RBAC: Adding query filter params (name=network) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:49.512676 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfc06f4-53f9-4ee1-a0eb-824fc7e7c733 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:49.514217 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfc06f4-53f9-4ee1-a0eb-824fc7e7c733 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:49.519292 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 395/776] 10.222.0.82 () {56 vars in 1251 bytes} [Thu Aug 18 16:54:49 2022] GET /identity/v3/endpoints?service_id=393d81dbb8af4b82a053e34bc06622c3&interface=public®ion_id=RegionOne => generated 194 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:49.520696 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee2cd20-81ce-4e2f-b6a8-f26d0de7c751 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:49.520988 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee2cd20-81ce-4e2f-b6a8-f26d0de7c751 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:49.521260 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee2cd20-81ce-4e2f-b6a8-f26d0de7c751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:49.523316 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 382/777] 10.222.0.82 () {56 vars in 1112 bytes} [Thu Aug 18 16:54:49 2022] GET /identity/v3/services?name=network => generated 124 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:49.525037 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfc06f4-53f9-4ee1-a0eb-824fc7e7c733 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:49.541256 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-1ee2cd20-81ce-4e2f-b6a8-f26d0de7c751 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:54:49.542456 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-1ee2cd20-81ce-4e2f-b6a8-f26d0de7c751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:54:49.586623 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:49.615402 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 396/778] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:54:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 16:54:49.616497 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee2cd20-81ce-4e2f-b6a8-f26d0de7c751 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:49.661449 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfc06f4-53f9-4ee1-a0eb-824fc7e7c733 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:49.698116 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b282e014-97ab-42b5-9824-1bf785ec9902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:49.698982 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282e014-97ab-42b5-9824-1bf785ec9902 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:49.699412 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282e014-97ab-42b5-9824-1bf785ec9902 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:49.699696 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282e014-97ab-42b5-9824-1bf785ec9902 None admin] PATH_INFO: `/v3/services` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:49.700829 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b282e014-97ab-42b5-9824-1bf785ec9902 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:49.701681 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b282e014-97ab-42b5-9824-1bf785ec9902 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:49.710938 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 383/779] 10.222.0.82 () {56 vars in 1087 bytes} [Thu Aug 18 16:54:49 2022] GET /identity/v3/services => generated 772 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:54:49.710938 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b860a8-c671-41e0-9011-ad9f3e2970c7 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:49.710938 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b860a8-c671-41e0-9011-ad9f3e2970c7 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:49.710938 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b860a8-c671-41e0-9011-ad9f3e2970c7 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:49.715200 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 384/780] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:54:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:54:49.716479 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6c77f7-57c0-429c-abc1-f069c4446b9b None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:49.716763 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6c77f7-57c0-429c-abc1-f069c4446b9b None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:49.717037 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6c77f7-57c0-429c-abc1-f069c4446b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:49.723899 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:49.741402 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-ed6c77f7-57c0-429c-abc1-f069c4446b9b None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:54:49.741402 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-ed6c77f7-57c0-429c-abc1-f069c4446b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:54:49.764664 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee2cd20-81ce-4e2f-b6a8-f26d0de7c751 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:49.806861 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36c9bf8-93a4-41da-8da0-0e7c0870cda7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:49.807817 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c9bf8-93a4-41da-8da0-0e7c0870cda7 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:49.809393 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c9bf8-93a4-41da-8da0-0e7c0870cda7 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:49.810015 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c9bf8-93a4-41da-8da0-0e7c0870cda7 None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:49.810845 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36c9bf8-93a4-41da-8da0-0e7c0870cda7 None admin] RBAC: Adding query filter params (type=network) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:49.811307 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36c9bf8-93a4-41da-8da0-0e7c0870cda7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:49.813214 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36c9bf8-93a4-41da-8da0-0e7c0870cda7 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:49.819891 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 397/781] 10.222.0.82 () {56 vars in 1112 bytes} [Thu Aug 18 16:54:49 2022] GET /identity/v3/services?type=network => generated 350 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:49.826226 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36c9bf8-93a4-41da-8da0-0e7c0870cda7 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:49.879679 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:49.929357 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 385/782] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:54:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 16:54:49.934994 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6c77f7-57c0-429c-abc1-f069c4446b9b None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:49.980509 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36c9bf8-93a4-41da-8da0-0e7c0870cda7 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:50.030189 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56edae98-822b-491a-923c-3e91c99c1b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:50.031834 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56edae98-822b-491a-923c-3e91c99c1b72 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:50.032347 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56edae98-822b-491a-923c-3e91c99c1b72 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:50.032826 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56edae98-822b-491a-923c-3e91c99c1b72 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:50.033728 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56edae98-822b-491a-923c-3e91c99c1b72 None admin] RBAC: Adding query filter params (interface=public, service_id=c4593407ddaf40aa9f8075e04573219f, region_id=RegionOne) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:50.034838 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56edae98-822b-491a-923c-3e91c99c1b72 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:50.036584 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56edae98-822b-491a-923c-3e91c99c1b72 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:50.045427 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 398/783] 10.222.0.82 () {56 vars in 1251 bytes} [Thu Aug 18 16:54:49 2022] GET /identity/v3/endpoints?service_id=c4593407ddaf40aa9f8075e04573219f&interface=public®ion_id=RegionOne => generated 194 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:50.050394 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56edae98-822b-491a-923c-3e91c99c1b72 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:50.050589 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6c77f7-57c0-429c-abc1-f069c4446b9b None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:50.086623 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:50.096311 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c458b78d-b865-49a5-911e-43bc46596f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:50.096887 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c458b78d-b865-49a5-911e-43bc46596f71 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:50.097123 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c458b78d-b865-49a5-911e-43bc46596f71 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:50.097393 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c458b78d-b865-49a5-911e-43bc46596f71 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:50.109201 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c458b78d-b865-49a5-911e-43bc46596f71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:54:50.113914 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-c458b78d-b865-49a5-911e-43bc46596f71 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 16:54:50.113914 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:50.113914 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:50.113914 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:50.113914 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:50.113914 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:50.113914 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:50.113914 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:50.113914 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:50.113914 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:50.113914 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:50.113914 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:50.113914 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:54:50.113914 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:54:50.113914 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:54:50.113914 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:54:50.114796 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:54:50.114796 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:54:50.114796 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:54:50.114796 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:54:50.114796 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:50.114796 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:50.114796 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:50.114796 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:50.114796 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:50.114796 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:50.114796 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:50.114796 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:50.114796 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:50.114796 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:50.114796 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:50.114796 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:50.115695 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:50.115695 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:50.115695 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:50.115695 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:50.115695 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:54:50.115695 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:54:50.115695 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:54:50.115695 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:54:50.115695 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:54:50.115695 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:54:50.115695 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 16:54:50.115695 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:50.120217 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 386/784] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 16:54:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 16:54:50.131011 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c458b78d-b865-49a5-911e-43bc46596f71 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:50.229781 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56edae98-822b-491a-923c-3e91c99c1b72 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:50.231895 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:50.271229 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c669f896-6a53-4620-be00-2643d73cd329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:50.271848 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c669f896-6a53-4620-be00-2643d73cd329 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:50.272127 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c669f896-6a53-4620-be00-2643d73cd329 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:50.272402 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c669f896-6a53-4620-be00-2643d73cd329 None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:50.273125 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c669f896-6a53-4620-be00-2643d73cd329 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:50.274607 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c669f896-6a53-4620-be00-2643d73cd329 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:50.281243 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 399/785] 10.222.0.82 () {56 vars in 1087 bytes} [Thu Aug 18 16:54:50 2022] GET /identity/v3/services => generated 772 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:54:50.282168 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c669f896-6a53-4620-be00-2643d73cd329 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:50.320091 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c458b78d-b865-49a5-911e-43bc46596f71 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:50.397570 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a15f26-efae-4860-b549-705c6f3eb34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:50.399536 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a15f26-efae-4860-b549-705c6f3eb34c None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:50.399825 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a15f26-efae-4860-b549-705c6f3eb34c None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:50.400101 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a15f26-efae-4860-b549-705c6f3eb34c None admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:50.400780 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a15f26-efae-4860-b549-705c6f3eb34c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:50.401085 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a15f26-efae-4860-b549-705c6f3eb34c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:50.402547 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a15f26-efae-4860-b549-705c6f3eb34c None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:50.422824 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 387/786] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:54:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:50.429591 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a15f26-efae-4860-b549-705c6f3eb34c None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:50.485885 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c669f896-6a53-4620-be00-2643d73cd329 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:50.501723 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:50.525678 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e62f1b2-a73b-4d0f-b3d6-68090ff767e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:50.526321 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e62f1b2-a73b-4d0f-b3d6-68090ff767e4 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:50.526603 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e62f1b2-a73b-4d0f-b3d6-68090ff767e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:50.526928 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e62f1b2-a73b-4d0f-b3d6-68090ff767e4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:50.541733 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e62f1b2-a73b-4d0f-b3d6-68090ff767e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:54:50.549879 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-1e62f1b2-a73b-4d0f-b3d6-68090ff767e4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 16:54:50.549879 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:50.549879 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:50.549879 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:50.549879 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:50.549879 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:50.549879 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:50.549879 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:50.549879 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:50.549879 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:50.549879 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:50.549879 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:50.549879 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:54:50.549879 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:54:50.549879 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:54:50.549879 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:54:50.550769 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:54:50.550769 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:54:50.550769 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:54:50.550769 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:54:50.550769 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:50.550769 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:50.550769 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:50.550769 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:50.550769 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:50.550769 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:50.550769 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:50.550769 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:50.550769 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:50.550769 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:50.550769 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:50.550769 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:50.551631 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:50.551631 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:50.551631 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:50.551631 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:50.551631 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:54:50.551631 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:54:50.551631 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:54:50.551631 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:54:50.551631 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:54:50.551631 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:54:50.551631 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 16:54:50.551631 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:50.554407 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 400/787] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 16:54:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:50.563127 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e62f1b2-a73b-4d0f-b3d6-68090ff767e4 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:50.577340 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a15f26-efae-4860-b549-705c6f3eb34c None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:50.614050 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:50.654571 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322e2298-0cf7-4148-9cbb-655c6ffa835c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:50.656262 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322e2298-0cf7-4148-9cbb-655c6ffa835c None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:50.657393 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322e2298-0cf7-4148-9cbb-655c6ffa835c None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:50.657864 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322e2298-0cf7-4148-9cbb-655c6ffa835c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:50.675631 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322e2298-0cf7-4148-9cbb-655c6ffa835c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:50.677545 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322e2298-0cf7-4148-9cbb-655c6ffa835c None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:50.695040 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-322e2298-0cf7-4148-9cbb-655c6ffa835c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:50.695040 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:50.695040 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:50.695040 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:50.695040 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:50.695040 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:50.695040 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:50.695040 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:50.695040 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:50.695040 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:50.695040 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:50.695040 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:50.695040 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:54:50.695040 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:54:50.695040 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:54:50.695040 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:54:50.696016 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:50.696016 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:50.696016 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:50.696016 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:50.696016 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:50.696016 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:50.696016 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:50.696016 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:50.696016 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:50.696016 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:50.696016 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:50.696016 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:50.696016 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:50.696016 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:50.696016 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:50.696016 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:50.696016 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:54:50.696901 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:54:50.696901 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:50.696901 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:50.700473 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 388/788] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:54:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:50.706294 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322e2298-0cf7-4148-9cbb-655c6ffa835c None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:50.820287 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e62f1b2-a73b-4d0f-b3d6-68090ff767e4 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:50.824795 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322e2298-0cf7-4148-9cbb-655c6ffa835c None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:50.855396 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:50.872718 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c9d6ff-1977-4a08-938f-8a1fe53ca3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:50.873339 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9d6ff-1977-4a08-938f-8a1fe53ca3ba None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:50.873626 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9d6ff-1977-4a08-938f-8a1fe53ca3ba None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:50.873900 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9d6ff-1977-4a08-938f-8a1fe53ca3ba None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:50.874595 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c9d6ff-1977-4a08-938f-8a1fe53ca3ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:50.874867 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c9d6ff-1977-4a08-938f-8a1fe53ca3ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:50.877003 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dcbc21-e3aa-4eb3-9f75-cc490460b597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:50.877694 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dcbc21-e3aa-4eb3-9f75-cc490460b597 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:50.877977 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dcbc21-e3aa-4eb3-9f75-cc490460b597 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:50.878251 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dcbc21-e3aa-4eb3-9f75-cc490460b597 None admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:50.878893 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dcbc21-e3aa-4eb3-9f75-cc490460b597 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:50.879217 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dcbc21-e3aa-4eb3-9f75-cc490460b597 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:50.880599 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dcbc21-e3aa-4eb3-9f75-cc490460b597 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:50.882185 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c9d6ff-1977-4a08-938f-8a1fe53ca3ba None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:50.900249 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 401/789] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:54:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:54:50.903894 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 389/790] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:54:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:50.909405 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c9d6ff-1977-4a08-938f-8a1fe53ca3ba None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:50.909703 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dcbc21-e3aa-4eb3-9f75-cc490460b597 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:51.056910 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dcbc21-e3aa-4eb3-9f75-cc490460b597 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:51.099385 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:51.105665 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad24506e-597c-4bb0-a7a4-5b58136b83ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:51.106312 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad24506e-597c-4bb0-a7a4-5b58136b83ec None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:51.106595 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad24506e-597c-4bb0-a7a4-5b58136b83ec None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:51.106874 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad24506e-597c-4bb0-a7a4-5b58136b83ec None admin] PATH_INFO: `/v3/users/swift` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:51.114501 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad24506e-597c-4bb0-a7a4-5b58136b83ec None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:51.124688 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad24506e-597c-4bb0-a7a4-5b58136b83ec None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:51.126831 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c9d6ff-1977-4a08-938f-8a1fe53ca3ba None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:51.128527 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-ad24506e-597c-4bb0-a7a4-5b58136b83ec None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 18 16:54:51.128527 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:51.128527 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:51.128527 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:51.128527 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:51.128527 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:51.128527 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:51.128527 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:51.128527 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:51.128527 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:51.128527 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:51.128527 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:51.128527 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 16:54:51.128527 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 16:54:51.128527 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 16:54:51.128527 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 16:54:51.129407 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:51.129407 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:51.129407 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 16:54:51.129407 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:54:51.129407 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 16:54:51.129407 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:54:51.129407 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:51.129407 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:51.129407 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:51.129407 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:51.129407 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:51.129407 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:51.129407 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:51.129407 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:51.129407 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:51.129407 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:51.129407 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:51.129407 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:51.130304 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:51.130304 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:51.130304 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 16:54:51.130304 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 16:54:51.130304 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 16:54:51.130304 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 16:54:51.130304 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 16:54:51.130304 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 16:54:51.130304 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 18 16:54:51.130304 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:51.130304 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 390/791] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 16:54:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 16:54:51.142170 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad24506e-597c-4bb0-a7a4-5b58136b83ec None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:51.191315 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:51.215725 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2495c983-a398-4115-ab25-d586daa07057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:51.216395 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2495c983-a398-4115-ab25-d586daa07057 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:51.216685 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2495c983-a398-4115-ab25-d586daa07057 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:51.216966 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2495c983-a398-4115-ab25-d586daa07057 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:51.240921 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2495c983-a398-4115-ab25-d586daa07057 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:51.242385 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2495c983-a398-4115-ab25-d586daa07057 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:51.267022 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad24506e-597c-4bb0-a7a4-5b58136b83ec None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:51.276171 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-2495c983-a398-4115-ab25-d586daa07057 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:51.276171 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:51.276171 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:51.276171 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:51.276171 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:51.276171 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:51.276171 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:51.276171 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:51.276171 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:51.276171 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:51.276171 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:51.276171 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:51.276171 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:54:51.276171 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:54:51.276171 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:54:51.276171 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:54:51.277972 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:51.277972 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:51.277972 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:51.277972 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:51.277972 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:51.277972 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:51.277972 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:51.277972 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:51.277972 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:51.277972 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:51.277972 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:51.277972 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:51.277972 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:51.277972 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:51.277972 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:51.277972 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:51.277972 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:54:51.279435 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:54:51.279435 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:51.279435 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:51.288111 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 402/792] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:54:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:51.293617 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2495c983-a398-4115-ab25-d586daa07057 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:51.317491 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b09a6e4-e2e3-43fc-8a50-4fa9dd6fd40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:51.318117 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b09a6e4-e2e3-43fc-8a50-4fa9dd6fd40d None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:51.318401 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b09a6e4-e2e3-43fc-8a50-4fa9dd6fd40d None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:51.318677 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b09a6e4-e2e3-43fc-8a50-4fa9dd6fd40d None admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:51.319442 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b09a6e4-e2e3-43fc-8a50-4fa9dd6fd40d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:51.319754 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b09a6e4-e2e3-43fc-8a50-4fa9dd6fd40d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:51.321198 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b09a6e4-e2e3-43fc-8a50-4fa9dd6fd40d None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:51.363854 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 391/793] 10.222.0.82 () {56 vars in 1135 bytes} [Thu Aug 18 16:54:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:51.367399 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b09a6e4-e2e3-43fc-8a50-4fa9dd6fd40d None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:51.472013 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:51.481449 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2495c983-a398-4115-ab25-d586daa07057 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:51.488847 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b09a6e4-e2e3-43fc-8a50-4fa9dd6fd40d None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:51.520601 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a48f84-32cc-4e54-9635-036eb74e2312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:51.521226 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a48f84-32cc-4e54-9635-036eb74e2312 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:51.521511 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a48f84-32cc-4e54-9635-036eb74e2312 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:51.521789 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a48f84-32cc-4e54-9635-036eb74e2312 None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:51.522550 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a48f84-32cc-4e54-9635-036eb74e2312 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:51.522992 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a48f84-32cc-4e54-9635-036eb74e2312 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:51.524475 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a48f84-32cc-4e54-9635-036eb74e2312 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:51.528118 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305fbb69-a615-4054-bd2d-21a339e293bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:51.528767 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305fbb69-a615-4054-bd2d-21a339e293bd None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:51.529053 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305fbb69-a615-4054-bd2d-21a339e293bd None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:51.529331 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305fbb69-a615-4054-bd2d-21a339e293bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:51.534477 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:51.543985 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305fbb69-a615-4054-bd2d-21a339e293bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:51.543985 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305fbb69-a615-4054-bd2d-21a339e293bd None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:51.554984 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 403/794] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:54:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:54:51.554984 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c8437-127a-420d-9766-06d6f6b1bd0a None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:51.554984 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c8437-127a-420d-9766-06d6f6b1bd0a None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:51.554984 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c8437-127a-420d-9766-06d6f6b1bd0a None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:51.554984 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 404/795] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:54:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:54:51.554984 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c8437-127a-420d-9766-06d6f6b1bd0a None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:51.561039 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-305fbb69-a615-4054-bd2d-21a339e293bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:51.561039 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:51.561039 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:51.561039 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:51.561039 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:51.561039 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:51.561039 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:51.561039 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:51.561039 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:51.561039 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:51.561039 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:51.561039 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:51.561039 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:54:51.561039 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:54:51.561039 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:54:51.561039 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:54:51.561949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:51.561949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:51.561949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:51.561949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:51.561949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:51.561949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:51.561949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:51.561949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:51.561949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:51.561949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:51.561949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:51.561949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:51.561949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:51.561949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:51.561949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:51.561949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:51.561949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:54:51.562816 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:54:51.562816 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:51.562816 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:51.562816 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 392/796] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:54:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:51.562816 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5e40b9-ccaa-487b-9e9e-cad973ed6b41 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:51.563154 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5e40b9-ccaa-487b-9e9e-cad973ed6b41 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:51.563224 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5e40b9-ccaa-487b-9e9e-cad973ed6b41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:51.582143 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-cd5e40b9-ccaa-487b-9e9e-cad973ed6b41 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:54:51.583374 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-cd5e40b9-ccaa-487b-9e9e-cad973ed6b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:54:51.659004 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 393/797] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:54:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 16:54:51.659004 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5e40b9-ccaa-487b-9e9e-cad973ed6b41 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:51.688659 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c8437-127a-420d-9766-06d6f6b1bd0a None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:51.722553 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:51.759392 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5e40b9-ccaa-487b-9e9e-cad973ed6b41 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:51.796435 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12227c94-4c2e-435a-8c35-38cd509e659b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:51.797052 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12227c94-4c2e-435a-8c35-38cd509e659b None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:51.797338 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12227c94-4c2e-435a-8c35-38cd509e659b None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:51.797615 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12227c94-4c2e-435a-8c35-38cd509e659b None admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:51.798261 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12227c94-4c2e-435a-8c35-38cd509e659b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:51.798565 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12227c94-4c2e-435a-8c35-38cd509e659b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:51.799637 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10fcb31-0afb-43a9-b8b0-904c7ede1a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:51.800075 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12227c94-4c2e-435a-8c35-38cd509e659b None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:51.800293 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10fcb31-0afb-43a9-b8b0-904c7ede1a83 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:51.800577 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10fcb31-0afb-43a9-b8b0-904c7ede1a83 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:51.800856 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10fcb31-0afb-43a9-b8b0-904c7ede1a83 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:51.812978 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 394/798] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:54:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:51.818145 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10fcb31-0afb-43a9-b8b0-904c7ede1a83 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:51.819649 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10fcb31-0afb-43a9-b8b0-904c7ede1a83 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:51.824624 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12227c94-4c2e-435a-8c35-38cd509e659b None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:51.832492 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-e10fcb31-0afb-43a9-b8b0-904c7ede1a83 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 18 16:54:51.832492 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:51.832492 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:51.832492 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:51.832492 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:51.832492 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:51.832492 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:51.832492 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:51.832492 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:51.832492 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:51.832492 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:51.832492 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:51.832492 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 16:54:51.832492 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 16:54:51.832492 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 16:54:51.832492 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 16:54:51.833731 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:51.833731 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:51.833731 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 16:54:51.833731 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:54:51.833731 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 16:54:51.833731 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:54:51.833731 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:51.833731 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:51.833731 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:51.833731 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:51.833731 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:51.833731 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:51.833731 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:51.833731 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:51.833731 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:51.833731 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:51.833731 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:51.833731 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:51.834640 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:51.834640 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:51.834640 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 16:54:51.834640 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 16:54:51.834640 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 16:54:51.834640 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 16:54:51.834640 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 16:54:51.834640 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 16:54:51.834640 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 18 16:54:51.834640 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:51.834640 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 405/799] 10.222.0.82 () {56 vars in 1093 bytes} [Thu Aug 18 16:54:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:51.842446 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10fcb31-0afb-43a9-b8b0-904c7ede1a83 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:51.935981 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:51.981966 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12227c94-4c2e-435a-8c35-38cd509e659b None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:52.008764 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:52.018036 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10fcb31-0afb-43a9-b8b0-904c7ede1a83 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:52.018342 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a04cabe-4e9e-4d44-bd04-28f8c6bd5e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:52.019011 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a04cabe-4e9e-4d44-bd04-28f8c6bd5e10 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:52.019294 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a04cabe-4e9e-4d44-bd04-28f8c6bd5e10 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:52.026267 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a04cabe-4e9e-4d44-bd04-28f8c6bd5e10 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:52.029164 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a04cabe-4e9e-4d44-bd04-28f8c6bd5e10 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:52.031244 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a04cabe-4e9e-4d44-bd04-28f8c6bd5e10 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:52.046748 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-1a04cabe-4e9e-4d44-bd04-28f8c6bd5e10 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:54:52.046748 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:52.046748 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:52.046748 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:52.046748 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:52.046748 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:52.046748 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:52.046748 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:52.046748 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:52.046748 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:52.046748 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:52.046748 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:52.046748 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 16:54:52.046748 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 16:54:52.046748 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 16:54:52.047683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 16:54:52.047683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:52.047683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:52.047683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:52.047683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:52.047683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:52.047683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:52.047683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:52.047683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:52.047683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:52.047683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:52.047683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:52.047683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:52.047683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:52.047683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:52.047683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:52.047683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:52.048942 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 16:54:52.048942 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 16:54:52.048942 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 16:54:52.048942 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 16:54:52.048942 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 16:54:52.048942 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 16:54:52.048942 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:54:52.048942 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:52.048942 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 395/800] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 16:54:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:52.049411 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a04cabe-4e9e-4d44-bd04-28f8c6bd5e10 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:52.061776 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3c36b4-caf5-43ba-821c-e1194e48cfcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:52.062739 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c36b4-caf5-43ba-821c-e1194e48cfcd None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:52.063253 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c36b4-caf5-43ba-821c-e1194e48cfcd None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:52.063781 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c36b4-caf5-43ba-821c-e1194e48cfcd None admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:52.064838 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3c36b4-caf5-43ba-821c-e1194e48cfcd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:52.065460 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3c36b4-caf5-43ba-821c-e1194e48cfcd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:52.067238 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3c36b4-caf5-43ba-821c-e1194e48cfcd None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:52.092572 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 406/801] 10.222.0.82 () {56 vars in 1133 bytes} [Thu Aug 18 16:54:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:52.093830 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3c36b4-caf5-43ba-821c-e1194e48cfcd None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:52.180169 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a04cabe-4e9e-4d44-bd04-28f8c6bd5e10 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:52.201734 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3c36b4-caf5-43ba-821c-e1194e48cfcd None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:52.224317 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:52.231013 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5a5fc8-141d-4645-912c-97001b6019c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:52.231698 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5a5fc8-141d-4645-912c-97001b6019c6 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:52.231991 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5a5fc8-141d-4645-912c-97001b6019c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:52.232272 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5a5fc8-141d-4645-912c-97001b6019c6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:52.245229 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5a5fc8-141d-4645-912c-97001b6019c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:54:52.248105 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924497a1-1b6e-420b-9289-3bc5139b20c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:52.248904 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924497a1-1b6e-420b-9289-3bc5139b20c7 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:52.249463 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924497a1-1b6e-420b-9289-3bc5139b20c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:52.249956 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924497a1-1b6e-420b-9289-3bc5139b20c7 None admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:52.250843 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-ad5a5fc8-141d-4645-912c-97001b6019c6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 16:54:52.250843 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:52.250843 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:52.250843 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:52.250843 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:52.250843 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:52.250843 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:52.250843 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:52.250843 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:52.250843 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:52.250843 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:52.250843 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:52.250843 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:54:52.250843 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:54:52.250843 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:54:52.250843 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:54:52.251745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:54:52.251745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:54:52.251745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:54:52.251745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:54:52.251745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:52.251745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:52.251745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:52.251745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:52.251745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:52.251745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:52.251745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:52.251745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:52.251745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:52.251745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:52.251745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:52.251745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:52.252545 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:52.252545 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:52.252545 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:52.252545 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:52.252545 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:54:52.252545 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:54:52.252545 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:54:52.252545 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:54:52.252545 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:54:52.252545 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:54:52.252545 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 16:54:52.252545 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:52.253147 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924497a1-1b6e-420b-9289-3bc5139b20c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:52.254020 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924497a1-1b6e-420b-9289-3bc5139b20c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:52.255701 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924497a1-1b6e-420b-9289-3bc5139b20c7 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:52.256923 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 396/802] 10.222.0.82 () {56 vars in 1102 bytes} [Thu Aug 18 16:54:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:52.257761 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5a5fc8-141d-4645-912c-97001b6019c6 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:52.275927 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 407/803] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 16:54:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:52.278570 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc9e5b2-2f79-4f4e-9376-9aac350cc823 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:52.279205 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc9e5b2-2f79-4f4e-9376-9aac350cc823 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:52.279751 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc9e5b2-2f79-4f4e-9376-9aac350cc823 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:52.281483 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 408/804] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:54:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:54:52.282442 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc9e5b2-2f79-4f4e-9376-9aac350cc823 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:52.310587 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:52.371774 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5a5fc8-141d-4645-912c-97001b6019c6 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:52.390928 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc9e5b2-2f79-4f4e-9376-9aac350cc823 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:52.414857 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac18fb1-b266-4ff5-8ae4-351b744ef02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:52.415504 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac18fb1-b266-4ff5-8ae4-351b744ef02e None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:52.415789 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac18fb1-b266-4ff5-8ae4-351b744ef02e None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:52.416067 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac18fb1-b266-4ff5-8ae4-351b744ef02e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:52.432414 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82958f22-39f3-434b-a18b-c30d5baf2c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:52.433116 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82958f22-39f3-434b-a18b-c30d5baf2c90 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:52.433392 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82958f22-39f3-434b-a18b-c30d5baf2c90 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:52.433662 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82958f22-39f3-434b-a18b-c30d5baf2c90 None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:52.434432 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82958f22-39f3-434b-a18b-c30d5baf2c90 None admin] RBAC: Adding query filter params (name=service) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:52.434732 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82958f22-39f3-434b-a18b-c30d5baf2c90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:52.436128 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82958f22-39f3-434b-a18b-c30d5baf2c90 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:52.444891 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 409/805] 10.222.0.82 () {56 vars in 1103 bytes} [Thu Aug 18 16:54:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:52.445494 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82958f22-39f3-434b-a18b-c30d5baf2c90 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:52.488729 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:52.492846 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac18fb1-b266-4ff5-8ae4-351b744ef02e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:52.494325 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac18fb1-b266-4ff5-8ae4-351b744ef02e None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:52.542844 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:52.556484 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82958f22-39f3-434b-a18b-c30d5baf2c90 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:52.592617 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d407aa0-8809-4b39-9b12-e6e700e087d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:52.593397 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d407aa0-8809-4b39-9b12-e6e700e087d6 None admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:52.593685 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d407aa0-8809-4b39-9b12-e6e700e087d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:52.593963 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d407aa0-8809-4b39-9b12-e6e700e087d6 None admin] PATH_INFO: `/v3/projects/2293c0aab61f44849d0d48b214cde7ea/users/1381673f81584f4b8084823e3aa99e2a/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:52.595915 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d407aa0-8809-4b39-9b12-e6e700e087d6 None admin] RBAC: Authorizing `identity:create_grant(project_id=2293c0aab61f44849d0d48b214cde7ea, user_id=1381673f81584f4b8084823e3aa99e2a, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:52.597351 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d407aa0-8809-4b39-9b12-e6e700e087d6 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:52.606641 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 397/806] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:54:52 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 16:54:52.612836 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a5e23f-d33c-425d-b81a-877aa86525ef None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:52.612836 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a5e23f-d33c-425d-b81a-877aa86525ef None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:52.612836 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a5e23f-d33c-425d-b81a-877aa86525ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:52.625489 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-48a5e23f-d33c-425d-b81a-877aa86525ef None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:54:52.626667 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-48a5e23f-d33c-425d-b81a-877aa86525ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:54:52.663235 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 410/807] 10.222.0.82 () {58 vars in 1439 bytes} [Thu Aug 18 16:54:52 2022] PUT /identity/v3/projects/2293c0aab61f44849d0d48b214cde7ea/users/1381673f81584f4b8084823e3aa99e2a/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 16:54:52.664590 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0840d7-7212-4983-8519-0047f7859aa8 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:52.664882 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0840d7-7212-4983-8519-0047f7859aa8 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:52.665154 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0840d7-7212-4983-8519-0047f7859aa8 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:52.666571 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 411/808] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:54:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:54:52.671910 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0840d7-7212-4983-8519-0047f7859aa8 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:52.695646 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 398/809] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:54:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 16:54:52.696436 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a5e23f-d33c-425d-b81a-877aa86525ef None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:52.806638 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:52.838155 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a5e23f-d33c-425d-b81a-877aa86525ef None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:52.868356 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:52.873515 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0840d7-7212-4983-8519-0047f7859aa8 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:52.885889 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95d020d-b05a-4bf5-9737-50e4c6540d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:52.887477 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d020d-b05a-4bf5-9737-50e4c6540d1e None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:52.887858 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d020d-b05a-4bf5-9737-50e4c6540d1e None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:52.888142 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d020d-b05a-4bf5-9737-50e4c6540d1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:52.902165 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95d020d-b05a-4bf5-9737-50e4c6540d1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:52.904742 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95d020d-b05a-4bf5-9737-50e4c6540d1e None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:52.912508 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f250cfa-bb0e-4529-abf5-02697470fc79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:52.913505 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f250cfa-bb0e-4529-abf5-02697470fc79 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:52.914115 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f250cfa-bb0e-4529-abf5-02697470fc79 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:52.914581 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f250cfa-bb0e-4529-abf5-02697470fc79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:52.921474 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-f95d020d-b05a-4bf5-9737-50e4c6540d1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:52.921474 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:52.921474 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:52.921474 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:52.921474 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:52.921474 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:52.921474 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:52.921474 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:52.921474 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:52.921474 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:52.921474 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:52.921474 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:52.921474 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:54:52.921474 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:54:52.921474 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:54:52.921474 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:54:52.922419 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:52.922419 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:52.922419 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:52.922419 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:52.922419 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:52.922419 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:52.922419 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:52.922419 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:52.922419 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:52.922419 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:52.922419 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:52.922419 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:52.922419 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:52.922419 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:52.922419 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:52.922419 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:52.922419 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:54:52.923369 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:54:52.923369 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:52.923369 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:52.923369 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 399/810] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:54:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:52.928223 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb93423-26d8-4943-9007-91ad7195e476 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:52.928223 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb93423-26d8-4943-9007-91ad7195e476 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:52.928223 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb93423-26d8-4943-9007-91ad7195e476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:52.933906 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f250cfa-bb0e-4529-abf5-02697470fc79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:52.936084 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f250cfa-bb0e-4529-abf5-02697470fc79 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:52.949389 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-acb93423-26d8-4943-9007-91ad7195e476 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:54:52.950643 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-acb93423-26d8-4943-9007-91ad7195e476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:54:52.953747 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-0f250cfa-bb0e-4529-abf5-02697470fc79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:52.953747 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:52.953747 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:52.953747 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:52.953747 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:52.953747 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:52.953747 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:52.953747 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:52.953747 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:52.953747 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:52.953747 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:52.953747 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:52.953747 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:54:52.953747 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:54:52.953747 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:54:52.953747 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:54:52.954765 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:52.954765 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:52.954765 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:52.954765 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:52.954765 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:52.954765 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:52.954765 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:52.954765 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:52.954765 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:52.954765 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:52.954765 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:52.954765 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:52.954765 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:52.954765 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:52.954765 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:52.954765 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:52.954765 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:54:52.955947 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:54:52.955947 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:52.955947 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:52.957735 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 412/811] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:54:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:52.958843 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f250cfa-bb0e-4529-abf5-02697470fc79 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:53.027434 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 400/812] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:54:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 16:54:53.027900 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb93423-26d8-4943-9007-91ad7195e476 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:53.070560 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f250cfa-bb0e-4529-abf5-02697470fc79 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:53.106405 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:53.115535 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221eff4-44a9-4df3-a9c3-116627841255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:53.116138 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221eff4-44a9-4df3-a9c3-116627841255 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:53.116417 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221eff4-44a9-4df3-a9c3-116627841255 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:53.116690 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221eff4-44a9-4df3-a9c3-116627841255 None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:53.117321 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6221eff4-44a9-4df3-a9c3-116627841255 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:53.117622 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6221eff4-44a9-4df3-a9c3-116627841255 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:53.119016 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6221eff4-44a9-4df3-a9c3-116627841255 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:53.132109 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 413/813] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:54:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:53.132944 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221eff4-44a9-4df3-a9c3-116627841255 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:53.148121 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:53.149139 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb93423-26d8-4943-9007-91ad7195e476 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:53.186101 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aa5622-c573-4478-afe8-59f6a45c7c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:53.186767 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aa5622-c573-4478-afe8-59f6a45c7c67 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:53.187089 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aa5622-c573-4478-afe8-59f6a45c7c67 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:53.187470 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aa5622-c573-4478-afe8-59f6a45c7c67 None admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:53.188002 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aa5622-c573-4478-afe8-59f6a45c7c67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:53.188307 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aa5622-c573-4478-afe8-59f6a45c7c67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:53.189704 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aa5622-c573-4478-afe8-59f6a45c7c67 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:53.203270 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 401/814] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:54:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:53.204176 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aa5622-c573-4478-afe8-59f6a45c7c67 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:53.242475 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221eff4-44a9-4df3-a9c3-116627841255 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:53.279069 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af35d08e-e947-4e61-b9e4-c6989976381b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:53.279962 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af35d08e-e947-4e61-b9e4-c6989976381b None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:53.280382 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af35d08e-e947-4e61-b9e4-c6989976381b None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:53.280809 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af35d08e-e947-4e61-b9e4-c6989976381b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:53.281523 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af35d08e-e947-4e61-b9e4-c6989976381b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:53.283228 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af35d08e-e947-4e61-b9e4-c6989976381b None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:53.295016 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-af35d08e-e947-4e61-b9e4-c6989976381b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 18 16:54:53.295016 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:53.295016 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 16:54:53.295016 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 16:54:53.295016 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 16:54:53.295016 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 16:54:53.295016 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 16:54:53.295016 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 16:54:53.295016 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 18 16:54:53.295016 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:53.295016 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 16:54:53.295016 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:53.295016 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:53.295016 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:53.295016 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:53.295016 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:53.295016 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:53.296176 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:53.296176 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:53.296176 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:53.296176 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:53.296176 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:53.296176 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:53.296176 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 16:54:53.296176 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 16:54:53.296176 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 16:54:53.296176 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 16:54:53.296176 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:53.296176 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:53.296176 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:53.296176 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:53.296176 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:53.296176 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:53.297116 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:53.297116 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:53.297116 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:53.297116 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:53.297116 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:53.297116 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:53.297116 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:53.297116 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:53.297116 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:53.297116 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:53.297116 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 16:54:53.297116 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 16:54:53.297116 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 18 16:54:53.297116 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:53.304061 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 414/815] 10.222.0.82 () {56 vars in 1129 bytes} [Thu Aug 18 16:54:53 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:53.305030 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af35d08e-e947-4e61-b9e4-c6989976381b None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:53.310872 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aa5622-c573-4478-afe8-59f6a45c7c67 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:53.328870 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:53.346010 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a94fec-cbc3-421a-a31f-34b7fcdd5a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:53.346643 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a94fec-cbc3-421a-a31f-34b7fcdd5a7e None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:53.346947 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a94fec-cbc3-421a-a31f-34b7fcdd5a7e None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:53.347225 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a94fec-cbc3-421a-a31f-34b7fcdd5a7e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:53.353830 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a94fec-cbc3-421a-a31f-34b7fcdd5a7e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:53.355916 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a94fec-cbc3-421a-a31f-34b7fcdd5a7e None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:53.367894 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-22a94fec-cbc3-421a-a31f-34b7fcdd5a7e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:54:53.367894 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:53.367894 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:53.367894 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:53.367894 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:53.367894 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:53.367894 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:53.367894 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:53.367894 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:53.367894 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:53.367894 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:53.367894 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:53.367894 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 16:54:53.367894 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 16:54:53.367894 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 16:54:53.368784 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 16:54:53.368784 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:53.368784 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:53.368784 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:53.368784 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:53.368784 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:53.368784 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:53.368784 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:53.368784 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:53.368784 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:53.368784 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:53.368784 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:53.368784 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:53.368784 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:53.368784 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:53.368784 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:53.368784 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:53.369690 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 16:54:53.369690 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 16:54:53.369690 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 16:54:53.369690 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 16:54:53.369690 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 16:54:53.369690 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 16:54:53.369690 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:54:53.369690 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:53.369690 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 402/816] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 16:54:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:53.370169 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a94fec-cbc3-421a-a31f-34b7fcdd5a7e None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:53.395128 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:53.419923 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af35d08e-e947-4e61-b9e4-c6989976381b None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:53.457582 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7095dd20-50e5-45c6-8869-ade95b4d0180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:53.458230 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7095dd20-50e5-45c6-8869-ade95b4d0180 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:53.458511 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7095dd20-50e5-45c6-8869-ade95b4d0180 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:53.458780 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7095dd20-50e5-45c6-8869-ade95b4d0180 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:53.468085 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7095dd20-50e5-45c6-8869-ade95b4d0180 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:53.469557 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7095dd20-50e5-45c6-8869-ade95b4d0180 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:53.482244 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-7095dd20-50e5-45c6-8869-ade95b4d0180 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 18 16:54:53.482244 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:53.482244 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:53.482244 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:53.482244 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:53.482244 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:53.482244 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:53.482244 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:53.482244 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:53.482244 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:53.482244 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:53.482244 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:53.482244 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 16:54:53.482244 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 16:54:53.482244 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 16:54:53.482244 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 16:54:53.483538 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:53.483538 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:53.483538 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 16:54:53.483538 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:54:53.483538 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 16:54:53.483538 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:54:53.483538 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:53.483538 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:53.483538 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:53.483538 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:53.483538 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:53.483538 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:53.483538 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:53.483538 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:53.483538 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:53.483538 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:53.483538 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:53.483538 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:53.484467 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:53.484467 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:53.484467 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 16:54:53.484467 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 16:54:53.484467 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 16:54:53.484467 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 16:54:53.484467 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 16:54:53.484467 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 16:54:53.484467 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 18 16:54:53.484467 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:53.487220 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 415/817] 10.222.0.82 () {56 vars in 1117 bytes} [Thu Aug 18 16:54:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:53.488323 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7095dd20-50e5-45c6-8869-ade95b4d0180 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:53.598697 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a94fec-cbc3-421a-a31f-34b7fcdd5a7e None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:53.601151 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7095dd20-50e5-45c6-8869-ade95b4d0180 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:53.634420 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3a8422-9b3a-4ae8-b6af-f3fc0814bdc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:53.635049 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a8422-9b3a-4ae8-b6af-f3fc0814bdc8 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:53.635329 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a8422-9b3a-4ae8-b6af-f3fc0814bdc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:53.635594 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a8422-9b3a-4ae8-b6af-f3fc0814bdc8 None admin] PATH_INFO: `/v3/services` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:53.636382 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3a8422-9b3a-4ae8-b6af-f3fc0814bdc8 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:53.636680 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3a8422-9b3a-4ae8-b6af-f3fc0814bdc8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:53.638042 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3a8422-9b3a-4ae8-b6af-f3fc0814bdc8 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:53.638143 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38a5936-3f89-4c4b-878f-e821df231e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:53.638710 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a5936-3f89-4c4b-878f-e821df231e55 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:53.639029 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a5936-3f89-4c4b-878f-e821df231e55 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:53.639289 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a5936-3f89-4c4b-878f-e821df231e55 None admin] PATH_INFO: `/v3/services/network` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:53.639870 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38a5936-3f89-4c4b-878f-e821df231e55 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:53.641195 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38a5936-3f89-4c4b-878f-e821df231e55 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:53.642810 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 403/818] 10.222.0.82 () {56 vars in 1124 bytes} [Thu Aug 18 16:54:53 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:53.644405 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:53.648111 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3a8422-9b3a-4ae8-b6af-f3fc0814bdc8 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:53.651050 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-c38a5936-3f89-4c4b-878f-e821df231e55 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 18 16:54:53.651050 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:53.651050 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 16:54:53.651050 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 16:54:53.651050 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 16:54:53.651050 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 16:54:53.651050 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 16:54:53.651050 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 16:54:53.651050 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 18 16:54:53.651050 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:53.651050 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 16:54:53.651050 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:53.651050 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:53.651050 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:53.651050 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:53.651050 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:53.651050 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:53.652591 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:53.652591 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:53.652591 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:53.652591 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:53.652591 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:53.652591 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:53.652591 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 16:54:53.652591 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 16:54:53.652591 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 16:54:53.652591 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 16:54:53.652591 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:53.652591 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:53.652591 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:53.652591 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:53.652591 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:53.652591 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:53.653399 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:53.653399 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:53.653399 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:53.653399 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:53.653399 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:53.653399 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:53.653399 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:53.653399 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:53.653399 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:53.653399 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:53.653399 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 16:54:53.653399 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 16:54:53.653399 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 18 16:54:53.653399 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:53.657424 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 416/819] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 16:54:53 2022] GET /identity/v3/services/network => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:53.658453 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38a5936-3f89-4c4b-878f-e821df231e55 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:53.694445 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:53.752729 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3a8422-9b3a-4ae8-b6af-f3fc0814bdc8 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:53.771533 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38a5936-3f89-4c4b-878f-e821df231e55 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:53.789286 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e246a4b6-a5f7-40ed-b121-d5c636ab0c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:53.789897 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e246a4b6-a5f7-40ed-b121-d5c636ab0c96 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:53.790175 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e246a4b6-a5f7-40ed-b121-d5c636ab0c96 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:53.790444 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e246a4b6-a5f7-40ed-b121-d5c636ab0c96 None admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:53.791157 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e246a4b6-a5f7-40ed-b121-d5c636ab0c96 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:53.791465 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e246a4b6-a5f7-40ed-b121-d5c636ab0c96 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:53.792958 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e246a4b6-a5f7-40ed-b121-d5c636ab0c96 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:53.805556 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 404/820] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 16:54:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:53.806732 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e246a4b6-a5f7-40ed-b121-d5c636ab0c96 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:53.819649 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cb7370-acb6-4201-a027-740ca01e81ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:53.820269 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb7370-acb6-4201-a027-740ca01e81ff None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:53.820551 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb7370-acb6-4201-a027-740ca01e81ff None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:53.820944 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb7370-acb6-4201-a027-740ca01e81ff None admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:53.821675 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cb7370-acb6-4201-a027-740ca01e81ff None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:53.821986 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cb7370-acb6-4201-a027-740ca01e81ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:53.824039 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cb7370-acb6-4201-a027-740ca01e81ff None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:53.843766 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 417/821] 10.222.0.82 () {56 vars in 1149 bytes} [Thu Aug 18 16:54:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:53.844426 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cb7370-acb6-4201-a027-740ca01e81ff None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:53.905430 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:53.920457 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e246a4b6-a5f7-40ed-b121-d5c636ab0c96 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:53.922520 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:53.946708 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cb7370-acb6-4201-a027-740ca01e81ff None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:53.955037 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54030912-43f3-4d63-91ab-84b6c9d4cc9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:53.955651 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54030912-43f3-4d63-91ab-84b6c9d4cc9b None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:53.955924 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54030912-43f3-4d63-91ab-84b6c9d4cc9b None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:53.956199 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54030912-43f3-4d63-91ab-84b6c9d4cc9b None admin] PATH_INFO: `/v3/services` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:53.956819 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54030912-43f3-4d63-91ab-84b6c9d4cc9b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:53.957121 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54030912-43f3-4d63-91ab-84b6c9d4cc9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:53.958501 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54030912-43f3-4d63-91ab-84b6c9d4cc9b None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:53.968504 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 405/822] 10.222.0.82 () {56 vars in 1124 bytes} [Thu Aug 18 16:54:53 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:53.968504 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54030912-43f3-4d63-91ab-84b6c9d4cc9b None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:53.994041 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9c8d62-1296-4063-bceb-a90476204a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:53.994743 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c8d62-1296-4063-bceb-a90476204a01 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:53.995291 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c8d62-1296-4063-bceb-a90476204a01 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:53.995637 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c8d62-1296-4063-bceb-a90476204a01 None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:53.996685 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9c8d62-1296-4063-bceb-a90476204a01 None admin] RBAC: Adding query filter params (name=network) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:53.997258 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9c8d62-1296-4063-bceb-a90476204a01 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:53.998634 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9c8d62-1296-4063-bceb-a90476204a01 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:54.008672 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 418/823] 10.222.0.82 () {56 vars in 1112 bytes} [Thu Aug 18 16:54:53 2022] GET /identity/v3/services?name=network => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:54.008672 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9c8d62-1296-4063-bceb-a90476204a01 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:54.066843 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54030912-43f3-4d63-91ab-84b6c9d4cc9b None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:54.101900 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15455b1f-c4d7-4e7a-820c-d0209548175f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:54.102511 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15455b1f-c4d7-4e7a-820c-d0209548175f None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:54.102803 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15455b1f-c4d7-4e7a-820c-d0209548175f None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:54.103102 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15455b1f-c4d7-4e7a-820c-d0209548175f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:54.103825 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15455b1f-c4d7-4e7a-820c-d0209548175f None admin] RBAC: Adding query filter params (role.id=2c6966cfb10f4d72b0242e9fb5e13b17, scope.project.id=2293c0aab61f44849d0d48b214cde7ea, user.id=55723c756cae4e7ab636054db48de02d) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:54.104135 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15455b1f-c4d7-4e7a-820c-d0209548175f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:54.105630 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15455b1f-c4d7-4e7a-820c-d0209548175f None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:54.111758 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 406/824] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 16:54:53 2022] GET /identity/v3/role_assignments?user.id=55723c756cae4e7ab636054db48de02d&scope.project.id=2293c0aab61f44849d0d48b214cde7ea&role.id=2c6966cfb10f4d72b0242e9fb5e13b17 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:54.112360 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15455b1f-c4d7-4e7a-820c-d0209548175f None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:54.126193 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9c8d62-1296-4063-bceb-a90476204a01 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:54.147491 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:54.158353 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:54.171950 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca240bcc-f733-46fe-8580-edd154cbc4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:54.173364 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca240bcc-f733-46fe-8580-edd154cbc4be None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:54.173855 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca240bcc-f733-46fe-8580-edd154cbc4be None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:54.174355 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca240bcc-f733-46fe-8580-edd154cbc4be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:54.187441 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca240bcc-f733-46fe-8580-edd154cbc4be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:54.189302 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca240bcc-f733-46fe-8580-edd154cbc4be None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:54.208064 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-ca240bcc-f733-46fe-8580-edd154cbc4be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:54.208064 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:54.208064 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:54.208064 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:54.208064 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:54.208064 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:54.208064 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:54.208064 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:54.208064 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:54.208064 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:54.208064 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:54.208064 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:54.208064 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:54:54.208064 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:54:54.208064 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:54:54.208064 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:54:54.209046 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:54.209046 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:54.209046 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:54.209046 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:54.209046 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:54.209046 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:54.209046 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:54.209046 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:54.209046 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:54.209046 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:54.209046 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:54.209046 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:54.209046 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:54.209046 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:54.209046 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:54.209046 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:54.209046 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:54:54.209913 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:54:54.209913 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:54.209913 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:54.217767 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 419/825] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:54:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:54.219592 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca240bcc-f733-46fe-8580-edd154cbc4be None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:54.228635 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15455b1f-c4d7-4e7a-820c-d0209548175f None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:54.268924 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1099efc-0433-44c7-b159-4a9d9da92bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:54.269569 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1099efc-0433-44c7-b159-4a9d9da92bfa None admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:54.269850 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1099efc-0433-44c7-b159-4a9d9da92bfa None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:54.270122 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1099efc-0433-44c7-b159-4a9d9da92bfa None admin] PATH_INFO: `/v3/endpoints` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:54.270783 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1099efc-0433-44c7-b159-4a9d9da92bfa None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:54.272319 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1099efc-0433-44c7-b159-4a9d9da92bfa None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:54.334532 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca240bcc-f733-46fe-8580-edd154cbc4be None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:54.372845 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c9de18-53b0-4ea3-a4e0-cc0b4ee4cb7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:54.373549 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c9de18-53b0-4ea3-a4e0-cc0b4ee4cb7c None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:54.373898 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c9de18-53b0-4ea3-a4e0-cc0b4ee4cb7c None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:54.374242 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c9de18-53b0-4ea3-a4e0-cc0b4ee4cb7c None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:54.374938 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c9de18-53b0-4ea3-a4e0-cc0b4ee4cb7c None admin] RBAC: Adding query filter params (type=network) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:54.375306 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c9de18-53b0-4ea3-a4e0-cc0b4ee4cb7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:54.376728 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c9de18-53b0-4ea3-a4e0-cc0b4ee4cb7c None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:54.383828 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 407/826] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 16:54:54 2022] POST /identity/v3/endpoints => generated 346 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 16:54:54.384761 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1099efc-0433-44c7-b159-4a9d9da92bfa None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:54.386856 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 420/827] 10.222.0.82 () {56 vars in 1112 bytes} [Thu Aug 18 16:54:54 2022] GET /identity/v3/services?type=network => generated 350 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:54.392950 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c9de18-53b0-4ea3-a4e0-cc0b4ee4cb7c None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:54.446566 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:54.527970 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c9de18-53b0-4ea3-a4e0-cc0b4ee4cb7c None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:54.584795 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:54.592142 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6ba915-1817-40c7-8bb3-bd89566145c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:54.593019 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ba915-1817-40c7-8bb3-bd89566145c7 None admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:54.594421 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ba915-1817-40c7-8bb3-bd89566145c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:54.595326 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ba915-1817-40c7-8bb3-bd89566145c7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:54.596419 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6ba915-1817-40c7-8bb3-bd89566145c7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:54.599434 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6ba915-1817-40c7-8bb3-bd89566145c7 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:54.625171 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1099efc-0433-44c7-b159-4a9d9da92bfa None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:54.665223 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f9e849-b055-40bd-b86e-4b5b4e398760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:54.666121 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f9e849-b055-40bd-b86e-4b5b4e398760 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:54.666591 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f9e849-b055-40bd-b86e-4b5b4e398760 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:54.667678 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f9e849-b055-40bd-b86e-4b5b4e398760 None admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:54.668666 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f9e849-b055-40bd-b86e-4b5b4e398760 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:54.669242 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f9e849-b055-40bd-b86e-4b5b4e398760 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:54.670782 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f9e849-b055-40bd-b86e-4b5b4e398760 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:54.687306 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 408/828] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:54:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:54.690263 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f9e849-b055-40bd-b86e-4b5b4e398760 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:54.726224 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 421/829] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 16:54:54 2022] POST /identity/v3/endpoints => generated 337 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 16:54:54.736277 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4795b49-22ab-42e7-a4ce-03eeecf18bac None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:54.736896 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4795b49-22ab-42e7-a4ce-03eeecf18bac None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:54.737319 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4795b49-22ab-42e7-a4ce-03eeecf18bac None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:54.739263 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 422/830] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:54:54 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:54:54.746761 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0328b53-8ab5-4732-81b6-69ba145775d8 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:54.747123 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0328b53-8ab5-4732-81b6-69ba145775d8 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:54.747407 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0328b53-8ab5-4732-81b6-69ba145775d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:54.782847 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-c0328b53-8ab5-4732-81b6-69ba145775d8 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:54:54.783222 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-c0328b53-8ab5-4732-81b6-69ba145775d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:54:54.884372 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f9e849-b055-40bd-b86e-4b5b4e398760 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:54.937990 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 423/831] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:54:54 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 16:54:54.944108 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdfbb54-abe1-4b1c-a9ff-1a70f64adc0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:54.944939 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdfbb54-abe1-4b1c-a9ff-1a70f64adc0f None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:54.945220 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdfbb54-abe1-4b1c-a9ff-1a70f64adc0f None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:54.945489 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdfbb54-abe1-4b1c-a9ff-1a70f64adc0f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:54.953115 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdfbb54-abe1-4b1c-a9ff-1a70f64adc0f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:54.954559 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdfbb54-abe1-4b1c-a9ff-1a70f64adc0f None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:54.967705 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-2cdfbb54-abe1-4b1c-a9ff-1a70f64adc0f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:54:54.967705 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:54.967705 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:54.967705 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:54.967705 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:54.967705 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:54.967705 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:54.967705 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:54.967705 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:54.967705 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:54.967705 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:54.967705 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:54.967705 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 16:54:54.967705 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 16:54:54.967705 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 16:54:54.968571 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 16:54:54.968571 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:54.968571 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:54.968571 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:54.968571 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:54.968571 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:54.968571 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:54.968571 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:54.968571 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:54.968571 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:54.968571 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:54.968571 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:54.968571 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:54.968571 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:54.968571 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:54.968571 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:54.968571 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:54.969440 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 16:54:54.969440 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 16:54:54.969440 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 16:54:54.969440 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 16:54:54.969440 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 16:54:54.969440 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 16:54:54.969440 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:54:54.969440 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:54.969440 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 409/832] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 16:54:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 16:54:54.974529 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0328b53-8ab5-4732-81b6-69ba145775d8 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:54.995455 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:55.081172 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0328b53-8ab5-4732-81b6-69ba145775d8 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:55.124344 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fa941f-638c-4bc8-9be7-667b4ee0d085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:55.124959 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa941f-638c-4bc8-9be7-667b4ee0d085 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:55.125235 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa941f-638c-4bc8-9be7-667b4ee0d085 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:55.125501 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa941f-638c-4bc8-9be7-667b4ee0d085 None admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:55.126187 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fa941f-638c-4bc8-9be7-667b4ee0d085 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:55.126485 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fa941f-638c-4bc8-9be7-667b4ee0d085 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:55.127942 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fa941f-638c-4bc8-9be7-667b4ee0d085 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:55.140508 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 424/833] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 16:54:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:55.146543 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdfbb54-abe1-4b1c-a9ff-1a70f64adc0f None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:55.162319 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:55.279209 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdfbb54-abe1-4b1c-a9ff-1a70f64adc0f None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:55.279994 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fa941f-638c-4bc8-9be7-667b4ee0d085 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:55.316357 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2134fd10-e00d-424e-8d01-140a9828f451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:55.317210 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2134fd10-e00d-424e-8d01-140a9828f451 None admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:55.317555 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2134fd10-e00d-424e-8d01-140a9828f451 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:55.317895 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2134fd10-e00d-424e-8d01-140a9828f451 None admin] PATH_INFO: `/v3/projects/2293c0aab61f44849d0d48b214cde7ea/users/5de19a54ed104bcdb35ee4e5e793f1e7/roles/5e0f402e1af84b44a723ee0730315d74` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:55.334242 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2134fd10-e00d-424e-8d01-140a9828f451 None admin] RBAC: Authorizing `identity:create_grant(project_id=2293c0aab61f44849d0d48b214cde7ea, user_id=5de19a54ed104bcdb35ee4e5e793f1e7, role_id=5e0f402e1af84b44a723ee0730315d74)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:55.335914 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2134fd10-e00d-424e-8d01-140a9828f451 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:55.360673 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:55.384927 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fa941f-638c-4bc8-9be7-667b4ee0d085 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:55.418889 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4ed370-a393-494d-9b49-a1e2b3a51421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:55.419543 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4ed370-a393-494d-9b49-a1e2b3a51421 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:55.419823 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4ed370-a393-494d-9b49-a1e2b3a51421 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:55.420096 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4ed370-a393-494d-9b49-a1e2b3a51421 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:55.431868 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4ed370-a393-494d-9b49-a1e2b3a51421 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:54:55.440131 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-7e4ed370-a393-494d-9b49-a1e2b3a51421 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 16:54:55.440131 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:55.440131 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:55.440131 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:55.440131 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:55.440131 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:55.440131 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:55.440131 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:55.440131 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:55.440131 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:55.440131 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:55.440131 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:55.440131 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:54:55.440131 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:54:55.440131 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:54:55.440131 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:54:55.441078 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:54:55.441078 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:54:55.441078 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:54:55.441078 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:54:55.441078 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:55.441078 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:55.441078 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:55.441078 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:55.441078 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:55.441078 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:55.441078 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:55.441078 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:55.441078 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:55.441078 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:55.441078 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:55.441078 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:55.441937 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:55.441937 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:55.441937 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:55.441937 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:55.441937 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:54:55.441937 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:54:55.441937 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:54:55.441937 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:54:55.441937 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:54:55.441937 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:54:55.441937 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 16:54:55.441937 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:55.443872 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 425/835] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 16:54:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 16:54:55.444325 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 410/835] 10.222.0.82 () {58 vars in 1439 bytes} [Thu Aug 18 16:54:55 2022] PUT /identity/v3/projects/2293c0aab61f44849d0d48b214cde7ea/users/5de19a54ed104bcdb35ee4e5e793f1e7/roles/5e0f402e1af84b44a723ee0730315d74 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 16:54:55.449984 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4ed370-a393-494d-9b49-a1e2b3a51421 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:55.570357 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4ed370-a393-494d-9b49-a1e2b3a51421 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:55.609089 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493d8a1b-6a58-4317-9db2-4a058507ca2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:55.609868 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493d8a1b-6a58-4317-9db2-4a058507ca2d None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:55.610252 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493d8a1b-6a58-4317-9db2-4a058507ca2d None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:55.610631 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493d8a1b-6a58-4317-9db2-4a058507ca2d None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:55.611443 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493d8a1b-6a58-4317-9db2-4a058507ca2d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:55.611851 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493d8a1b-6a58-4317-9db2-4a058507ca2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:55.613374 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493d8a1b-6a58-4317-9db2-4a058507ca2d None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:55.616055 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:55.624156 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 426/836] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:54:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:54:55.629998 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2134fd10-e00d-424e-8d01-140a9828f451 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:55.693421 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:55.742832 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2134fd10-e00d-424e-8d01-140a9828f451 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:55.784754 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e3f8f3-1237-47aa-ab80-d241bc1b364d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:55.785763 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3f8f3-1237-47aa-ab80-d241bc1b364d None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:55.786237 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3f8f3-1237-47aa-ab80-d241bc1b364d None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:55.786734 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3f8f3-1237-47aa-ab80-d241bc1b364d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:55.798192 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e3f8f3-1237-47aa-ab80-d241bc1b364d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:55.800025 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e3f8f3-1237-47aa-ab80-d241bc1b364d None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:55.816718 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-c0e3f8f3-1237-47aa-ab80-d241bc1b364d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:55.816718 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:55.816718 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:55.816718 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:55.816718 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:55.816718 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:55.816718 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:55.816718 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:55.816718 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:55.816718 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:55.816718 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:55.816718 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:55.816718 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:54:55.816718 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:54:55.816718 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:54:55.816718 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:54:55.817696 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:55.817696 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:55.817696 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:55.817696 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:55.817696 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:55.817696 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:55.817696 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:55.817696 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:55.817696 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:55.817696 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:55.817696 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:55.817696 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:55.817696 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:55.817696 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:55.817696 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:55.817696 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:55.817696 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:54:55.818549 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:54:55.818549 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:55.818549 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:55.820859 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 411/837] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:54:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:55.828022 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493d8a1b-6a58-4317-9db2-4a058507ca2d None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:55.938002 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493d8a1b-6a58-4317-9db2-4a058507ca2d None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:55.974296 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2afd2aa-e14a-4967-845b-515373fd0f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:55.974924 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2afd2aa-e14a-4967-845b-515373fd0f54 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:55.975218 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2afd2aa-e14a-4967-845b-515373fd0f54 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:55.975495 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2afd2aa-e14a-4967-845b-515373fd0f54 None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:55.976137 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2afd2aa-e14a-4967-845b-515373fd0f54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:55.976442 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2afd2aa-e14a-4967-845b-515373fd0f54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:55.977807 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2afd2aa-e14a-4967-845b-515373fd0f54 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:55.990022 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 427/838] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:54:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:55.996766 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e3f8f3-1237-47aa-ab80-d241bc1b364d None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:56.107579 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e3f8f3-1237-47aa-ab80-d241bc1b364d None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:56.116965 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:56.142428 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b295d5-b62a-4c9b-8300-07f5a947ef22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:56.143121 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b295d5-b62a-4c9b-8300-07f5a947ef22 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:56.143393 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b295d5-b62a-4c9b-8300-07f5a947ef22 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:56.143669 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b295d5-b62a-4c9b-8300-07f5a947ef22 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:56.152327 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b295d5-b62a-4c9b-8300-07f5a947ef22 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:56.153996 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b295d5-b62a-4c9b-8300-07f5a947ef22 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:56.166197 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-e8b295d5-b62a-4c9b-8300-07f5a947ef22 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 18 16:54:56.166197 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:56.166197 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:56.166197 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:56.166197 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:56.166197 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:56.166197 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:56.166197 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:56.166197 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:56.166197 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:56.166197 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:56.166197 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:56.166197 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 16:54:56.166197 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 16:54:56.166197 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 16:54:56.166197 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 16:54:56.167606 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:56.167606 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:56.167606 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 16:54:56.167606 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:54:56.167606 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 16:54:56.167606 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:54:56.167606 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:56.167606 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:56.167606 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:56.167606 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:56.167606 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:56.167606 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:56.167606 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:56.167606 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:56.167606 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:56.167606 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:56.167606 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:56.167606 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:56.168669 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:56.168669 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:56.168669 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 16:54:56.168669 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 16:54:56.168669 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 16:54:56.168669 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 16:54:56.168669 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 16:54:56.168669 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 16:54:56.168669 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 18 16:54:56.168669 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:56.169617 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 412/839] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 16:54:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 16:54:56.174996 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2afd2aa-e14a-4967-845b-515373fd0f54 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:56.236108 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8229ca77-2302-4400-b2cf-0c82de0313fd None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:56.236434 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8229ca77-2302-4400-b2cf-0c82de0313fd None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:56.236660 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8229ca77-2302-4400-b2cf-0c82de0313fd None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:56.238886 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 413/840] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:54:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 16:54:56.240705 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:56.246404 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009836b-a098-454a-ba99-a9a0ef14b677 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:56.246749 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009836b-a098-454a-ba99-a9a0ef14b677 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:56.247079 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009836b-a098-454a-ba99-a9a0ef14b677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:56.270329 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-6009836b-a098-454a-ba99-a9a0ef14b677 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:54:56.271478 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-6009836b-a098-454a-ba99-a9a0ef14b677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:54:56.289757 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2afd2aa-e14a-4967-845b-515373fd0f54 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:56.326328 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0754a8f3-bfce-4736-8491-cd9f0c5bd0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:56.327294 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0754a8f3-bfce-4736-8491-cd9f0c5bd0fc None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:56.327777 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0754a8f3-bfce-4736-8491-cd9f0c5bd0fc None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:56.328250 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0754a8f3-bfce-4736-8491-cd9f0c5bd0fc None admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:56.329170 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0754a8f3-bfce-4736-8491-cd9f0c5bd0fc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:56.329757 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0754a8f3-bfce-4736-8491-cd9f0c5bd0fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:56.331491 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0754a8f3-bfce-4736-8491-cd9f0c5bd0fc None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:56.356133 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 428/841] 10.222.0.82 () {56 vars in 1135 bytes} [Thu Aug 18 16:54:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:56.361904 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0754a8f3-bfce-4736-8491-cd9f0c5bd0fc None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:56.370960 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 414/842] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:54:56 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 16:54:56.473755 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0754a8f3-bfce-4736-8491-cd9f0c5bd0fc None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:56.491684 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:56.510007 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5cc83a-3353-42d5-af23-0e61abe7a849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:56.510993 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5cc83a-3353-42d5-af23-0e61abe7a849 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:56.511476 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5cc83a-3353-42d5-af23-0e61abe7a849 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:56.511969 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5cc83a-3353-42d5-af23-0e61abe7a849 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:56.582725 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5cc83a-3353-42d5-af23-0e61abe7a849 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:56.584535 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5cc83a-3353-42d5-af23-0e61abe7a849 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:56.586458 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2170cf-7bda-4311-9967-3ec720500d35 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:56.586745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2170cf-7bda-4311-9967-3ec720500d35 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:56.587118 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2170cf-7bda-4311-9967-3ec720500d35 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:56.589565 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 415/843] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:54:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 16:54:56.594253 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454ec406-48e6-4fd9-873f-560f1a653c79 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:56.594537 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454ec406-48e6-4fd9-873f-560f1a653c79 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:56.594819 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454ec406-48e6-4fd9-873f-560f1a653c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:56.612801 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-454ec406-48e6-4fd9-873f-560f1a653c79 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:54:56.613881 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-454ec406-48e6-4fd9-873f-560f1a653c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:54:56.697977 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 416/844] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:54:56 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 16:54:56.698711 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454ec406-48e6-4fd9-873f-560f1a653c79 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:56.731391 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:56.747159 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 429/845] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:54:56 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 16:54:56.749874 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5cc83a-3353-42d5-af23-0e61abe7a849 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:56.770932 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:56.808143 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454ec406-48e6-4fd9-873f-560f1a653c79 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:56.845126 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505dea5b-9753-4e88-ae0b-7844ad864e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:56.845798 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505dea5b-9753-4e88-ae0b-7844ad864e08 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:56.846081 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505dea5b-9753-4e88-ae0b-7844ad864e08 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:56.846352 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505dea5b-9753-4e88-ae0b-7844ad864e08 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:56.846956 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505dea5b-9753-4e88-ae0b-7844ad864e08 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:56.848352 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505dea5b-9753-4e88-ae0b-7844ad864e08 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:56.857678 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-505dea5b-9753-4e88-ae0b-7844ad864e08 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 18 16:54:56.857678 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:56.857678 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 16:54:56.857678 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 16:54:56.857678 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 16:54:56.857678 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 16:54:56.857678 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 16:54:56.857678 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 16:54:56.857678 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 18 16:54:56.857678 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:56.857678 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 16:54:56.857678 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:56.857678 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:56.857678 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:56.857678 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:56.857678 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:56.858586 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:56.858586 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:56.858586 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:56.858586 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:56.858586 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:56.858586 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:56.858586 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:56.858586 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 16:54:56.858586 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 16:54:56.858586 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 16:54:56.858586 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 16:54:56.858586 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:56.858586 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:56.858586 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:56.858586 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:56.858586 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:56.858586 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:56.859472 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:56.859472 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:56.859472 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:56.859472 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:56.859472 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:56.859472 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:56.859472 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:56.859472 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:56.859472 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:56.859472 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:56.859472 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 16:54:56.859472 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 16:54:56.859472 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 18 16:54:56.859472 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:56.864213 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 417/846] 10.222.0.82 () {56 vars in 1132 bytes} [Thu Aug 18 16:54:56 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:56.864595 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5cc83a-3353-42d5-af23-0e61abe7a849 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:56.864768 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505dea5b-9753-4e88-ae0b-7844ad864e08 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:56.900580 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89541cc4-5e9a-4b76-80bf-4227225912ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:56.902002 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89541cc4-5e9a-4b76-80bf-4227225912ef None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:56.902437 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89541cc4-5e9a-4b76-80bf-4227225912ef None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:56.902776 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89541cc4-5e9a-4b76-80bf-4227225912ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:56.913392 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89541cc4-5e9a-4b76-80bf-4227225912ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:56.914947 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89541cc4-5e9a-4b76-80bf-4227225912ef None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:56.931156 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-89541cc4-5e9a-4b76-80bf-4227225912ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:56.931156 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:56.931156 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:56.931156 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:56.931156 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:56.931156 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:56.931156 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:56.931156 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:56.931156 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:56.931156 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:56.931156 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:56.931156 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:56.931156 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:54:56.931156 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:54:56.931156 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:54:56.931156 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:54:56.933104 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:56.933104 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:56.933104 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:56.933104 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:56.933104 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:56.933104 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:56.933104 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:56.933104 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:56.933104 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:56.933104 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:56.933104 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:56.933104 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:56.933104 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:56.933104 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:56.933104 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:56.933104 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:56.933104 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:54:56.935422 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:54:56.935422 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:56.935422 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:56.942940 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 430/847] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:54:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 16:54:56.942940 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89541cc4-5e9a-4b76-80bf-4227225912ef None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:56.961726 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:56.972423 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505dea5b-9753-4e88-ae0b-7844ad864e08 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:57.008667 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e35f18-9bee-44e0-8e63-eee59bd567e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:57.009274 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e35f18-9bee-44e0-8e63-eee59bd567e2 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:57.009549 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e35f18-9bee-44e0-8e63-eee59bd567e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:57.009820 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e35f18-9bee-44e0-8e63-eee59bd567e2 None admin] PATH_INFO: `/v3/services` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:57.010441 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e35f18-9bee-44e0-8e63-eee59bd567e2 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:57.010835 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e35f18-9bee-44e0-8e63-eee59bd567e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:57.012849 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e35f18-9bee-44e0-8e63-eee59bd567e2 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:57.017486 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:57.022938 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 418/848] 10.222.0.82 () {56 vars in 1126 bytes} [Thu Aug 18 16:54:56 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:57.022938 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e35f18-9bee-44e0-8e63-eee59bd567e2 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:57.065908 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89541cc4-5e9a-4b76-80bf-4227225912ef None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:57.103922 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a865dee-e6c9-4a5e-9fcb-f79947dfd5ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:57.105213 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a865dee-e6c9-4a5e-9fcb-f79947dfd5ba None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:57.105499 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a865dee-e6c9-4a5e-9fcb-f79947dfd5ba None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:57.105773 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a865dee-e6c9-4a5e-9fcb-f79947dfd5ba None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:57.106412 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a865dee-e6c9-4a5e-9fcb-f79947dfd5ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:57.106715 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a865dee-e6c9-4a5e-9fcb-f79947dfd5ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:57.108126 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a865dee-e6c9-4a5e-9fcb-f79947dfd5ba None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:57.121404 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 431/849] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:54:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:54:57.121972 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a865dee-e6c9-4a5e-9fcb-f79947dfd5ba None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:57.132203 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e35f18-9bee-44e0-8e63-eee59bd567e2 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:57.166648 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a760c62a-208e-4669-babd-c36f112a03aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:57.167801 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a760c62a-208e-4669-babd-c36f112a03aa None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:57.168080 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a760c62a-208e-4669-babd-c36f112a03aa None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:57.168360 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a760c62a-208e-4669-babd-c36f112a03aa None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:57.168935 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a760c62a-208e-4669-babd-c36f112a03aa None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:57.170315 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a760c62a-208e-4669-babd-c36f112a03aa None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:57.179066 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-a760c62a-208e-4669-babd-c36f112a03aa None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 18 16:54:57.179066 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:57.179066 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 16:54:57.179066 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 16:54:57.179066 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 16:54:57.179066 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 16:54:57.179066 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 16:54:57.179066 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 16:54:57.179066 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 18 16:54:57.179066 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:57.179066 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 16:54:57.179066 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:57.179066 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:57.179066 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:57.179066 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:57.179066 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:57.179066 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:57.180007 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:57.180007 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:57.180007 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:57.180007 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:57.180007 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:57.180007 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:57.180007 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 16:54:57.180007 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 16:54:57.180007 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 16:54:57.180007 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 16:54:57.180007 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:57.180007 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:57.180007 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:57.180007 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:57.180007 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:57.180007 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:57.180835 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:57.180835 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:57.180835 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:57.180835 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:57.180835 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:57.180835 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:57.180835 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:57.180835 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:57.180835 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:57.180835 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:57.180835 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 16:54:57.180835 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 16:54:57.180835 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 18 16:54:57.180835 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:57.180835 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 419/850] 10.222.0.82 () {56 vars in 1114 bytes} [Thu Aug 18 16:54:57 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:57.181598 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a760c62a-208e-4669-babd-c36f112a03aa None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:57.205637 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:57.234605 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a865dee-e6c9-4a5e-9fcb-f79947dfd5ba None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:57.263848 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:57.273635 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f32e29c-d6cb-4637-b96f-ada91bcb1af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:57.274255 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f32e29c-d6cb-4637-b96f-ada91bcb1af4 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:57.274539 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f32e29c-d6cb-4637-b96f-ada91bcb1af4 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:57.274815 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f32e29c-d6cb-4637-b96f-ada91bcb1af4 None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:57.275477 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f32e29c-d6cb-4637-b96f-ada91bcb1af4 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:57.275851 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f32e29c-d6cb-4637-b96f-ada91bcb1af4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:57.279840 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f32e29c-d6cb-4637-b96f-ada91bcb1af4 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:57.287061 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 432/851] 10.222.0.82 () {56 vars in 1126 bytes} [Thu Aug 18 16:54:57 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:57.288008 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f32e29c-d6cb-4637-b96f-ada91bcb1af4 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:57.297766 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a760c62a-208e-4669-babd-c36f112a03aa None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:57.347000 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78bf215-05fd-4d88-93cc-accad6211609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:57.348293 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78bf215-05fd-4d88-93cc-accad6211609 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:57.348662 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78bf215-05fd-4d88-93cc-accad6211609 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:57.349027 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78bf215-05fd-4d88-93cc-accad6211609 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:57.360666 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78bf215-05fd-4d88-93cc-accad6211609 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:57.362652 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78bf215-05fd-4d88-93cc-accad6211609 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:57.379922 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-a78bf215-05fd-4d88-93cc-accad6211609 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:54:57.379922 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:57.379922 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:57.379922 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:57.379922 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:57.379922 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:57.379922 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:57.379922 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:57.379922 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:57.379922 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:57.379922 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:57.379922 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:57.379922 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 16:54:57.379922 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 16:54:57.379922 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 16:54:57.381254 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 16:54:57.381254 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:57.381254 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:57.381254 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:57.381254 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:57.381254 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:57.381254 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:57.381254 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:57.381254 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:57.381254 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:57.381254 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:57.381254 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:57.381254 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:57.381254 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:57.381254 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:57.381254 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:57.381254 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:57.382120 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 16:54:57.382120 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 16:54:57.382120 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 16:54:57.382120 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 16:54:57.382120 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 16:54:57.382120 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 16:54:57.382120 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:54:57.382120 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:57.386559 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 420/852] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 16:54:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:57.387526 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78bf215-05fd-4d88-93cc-accad6211609 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:57.416516 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f32e29c-d6cb-4637-b96f-ada91bcb1af4 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:57.457343 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1410a54-5d45-4e70-9979-802081b9e51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:57.458077 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1410a54-5d45-4e70-9979-802081b9e51a None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:57.458360 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1410a54-5d45-4e70-9979-802081b9e51a None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:57.458631 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1410a54-5d45-4e70-9979-802081b9e51a None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:57.459397 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1410a54-5d45-4e70-9979-802081b9e51a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:57.459786 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1410a54-5d45-4e70-9979-802081b9e51a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:57.461236 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1410a54-5d45-4e70-9979-802081b9e51a None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:57.471431 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 433/853] 10.222.0.82 () {56 vars in 1114 bytes} [Thu Aug 18 16:54:57 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:57.477419 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1410a54-5d45-4e70-9979-802081b9e51a None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:57.511108 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:57.543740 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:57.545180 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78bf215-05fd-4d88-93cc-accad6211609 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:57.589270 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1410a54-5d45-4e70-9979-802081b9e51a None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:57.589965 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72813ec7-e40e-41ae-9a92-75ccc4e269e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:57.590573 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72813ec7-e40e-41ae-9a92-75ccc4e269e9 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:57.590846 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72813ec7-e40e-41ae-9a92-75ccc4e269e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:57.591154 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72813ec7-e40e-41ae-9a92-75ccc4e269e9 None admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:57.591858 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72813ec7-e40e-41ae-9a92-75ccc4e269e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:57.592123 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72813ec7-e40e-41ae-9a92-75ccc4e269e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:57.593699 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72813ec7-e40e-41ae-9a92-75ccc4e269e9 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:57.607173 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 421/854] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 16:54:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:57.613026 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf6d20c-16a9-4be2-a3b0-933ea54dd2fb None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:57.613313 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf6d20c-16a9-4be2-a3b0-933ea54dd2fb None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:57.613586 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf6d20c-16a9-4be2-a3b0-933ea54dd2fb None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:57.619067 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 422/855] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:54:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:54:57.619067 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf6d20c-16a9-4be2-a3b0-933ea54dd2fb None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:57.634916 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6330e4f-b7b6-4c4f-be1b-8196fc2e6e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:57.635659 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6330e4f-b7b6-4c4f-be1b-8196fc2e6e94 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:57.635940 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6330e4f-b7b6-4c4f-be1b-8196fc2e6e94 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:57.636248 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6330e4f-b7b6-4c4f-be1b-8196fc2e6e94 None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:57.636921 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6330e4f-b7b6-4c4f-be1b-8196fc2e6e94 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:57.637228 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6330e4f-b7b6-4c4f-be1b-8196fc2e6e94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:57.638703 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6330e4f-b7b6-4c4f-be1b-8196fc2e6e94 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:57.643907 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 434/856] 10.222.0.82 () {56 vars in 1114 bytes} [Thu Aug 18 16:54:57 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:54:57.654630 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9e32bc-414f-478d-b18b-389cc94da6cc None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:57.655043 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9e32bc-414f-478d-b18b-389cc94da6cc None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:57.655314 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9e32bc-414f-478d-b18b-389cc94da6cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:57.691316 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-1d9e32bc-414f-478d-b18b-389cc94da6cc None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:54:57.692568 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-1d9e32bc-414f-478d-b18b-389cc94da6cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:54:57.767266 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf6d20c-16a9-4be2-a3b0-933ea54dd2fb None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:57.779631 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 435/857] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:54:57 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 16:54:57.811964 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316fad9b-8c44-4651-8f30-b9a36878cc76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:57.812984 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316fad9b-8c44-4651-8f30-b9a36878cc76 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:57.813535 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316fad9b-8c44-4651-8f30-b9a36878cc76 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:57.814006 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316fad9b-8c44-4651-8f30-b9a36878cc76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:57.814958 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316fad9b-8c44-4651-8f30-b9a36878cc76 None admin] RBAC: Adding query filter params (role.id=2c6966cfb10f4d72b0242e9fb5e13b17, scope.project.id=2293c0aab61f44849d0d48b214cde7ea, user.id=1381673f81584f4b8084823e3aa99e2a) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:57.815514 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316fad9b-8c44-4651-8f30-b9a36878cc76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:57.817207 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316fad9b-8c44-4651-8f30-b9a36878cc76 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:57.826233 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 423/858] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 16:54:57 2022] GET /identity/v3/role_assignments?user.id=1381673f81584f4b8084823e3aa99e2a&scope.project.id=2293c0aab61f44849d0d48b214cde7ea&role.id=2c6966cfb10f4d72b0242e9fb5e13b17 => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:58.069514 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9e32bc-414f-478d-b18b-389cc94da6cc None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:58.123325 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:58.174959 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9e32bc-414f-478d-b18b-389cc94da6cc None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:58.209774 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2658036b-39a3-46db-a76d-5f2529818aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:58.210409 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658036b-39a3-46db-a76d-5f2529818aad None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:58.210686 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658036b-39a3-46db-a76d-5f2529818aad None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:58.210976 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658036b-39a3-46db-a76d-5f2529818aad None admin] PATH_INFO: `/v3/roles/service` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:58.222814 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2658036b-39a3-46db-a76d-5f2529818aad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:54:58.227469 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-2658036b-39a3-46db-a76d-5f2529818aad None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 16:54:58.227469 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:58.227469 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:58.227469 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:58.227469 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:58.227469 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:58.227469 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:58.227469 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:58.227469 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:58.227469 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:58.227469 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:58.227469 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:58.227469 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:54:58.227469 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:54:58.227469 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:54:58.227469 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:54:58.228364 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:54:58.228364 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:54:58.228364 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:54:58.228364 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:54:58.228364 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:58.228364 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:58.228364 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:58.228364 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:58.228364 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:58.228364 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:58.228364 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:58.228364 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:58.228364 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:58.228364 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:58.228364 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:58.228364 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:58.229644 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:58.229644 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:58.229644 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:58.229644 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:58.229644 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:54:58.229644 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:54:58.229644 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:54:58.229644 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:54:58.229644 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:54:58.229644 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:54:58.229644 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 16:54:58.229644 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:58.229644 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 436/859] 10.222.0.82 () {56 vars in 1102 bytes} [Thu Aug 18 16:54:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:58.235695 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316fad9b-8c44-4651-8f30-b9a36878cc76 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:58.251274 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:58.340913 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316fad9b-8c44-4651-8f30-b9a36878cc76 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:58.377291 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d61ddf-7f08-47b0-aaa9-e6597d57106b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:58.377906 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d61ddf-7f08-47b0-aaa9-e6597d57106b None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:58.378186 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d61ddf-7f08-47b0-aaa9-e6597d57106b None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:58.378457 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d61ddf-7f08-47b0-aaa9-e6597d57106b None admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:58.379200 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d61ddf-7f08-47b0-aaa9-e6597d57106b None admin] RBAC: Adding query filter params (name=service) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:58.379503 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d61ddf-7f08-47b0-aaa9-e6597d57106b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:58.380926 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d61ddf-7f08-47b0-aaa9-e6597d57106b None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:58.390406 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 424/860] 10.222.0.82 () {56 vars in 1103 bytes} [Thu Aug 18 16:54:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:58.395976 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2658036b-39a3-46db-a76d-5f2529818aad None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:58.502609 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2658036b-39a3-46db-a76d-5f2529818aad None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:58.514410 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:58.537926 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c5247e-4648-4395-ba2b-a7c9ce7da68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:58.538559 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c5247e-4648-4395-ba2b-a7c9ce7da68a None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:58.538836 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c5247e-4648-4395-ba2b-a7c9ce7da68a None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:58.539143 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c5247e-4648-4395-ba2b-a7c9ce7da68a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:58.550305 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c5247e-4648-4395-ba2b-a7c9ce7da68a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:58.551769 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c5247e-4648-4395-ba2b-a7c9ce7da68a None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:58.567129 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-35c5247e-4648-4395-ba2b-a7c9ce7da68a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:58.567129 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:58.567129 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:58.567129 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:58.567129 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:58.567129 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:58.567129 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:58.567129 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:58.567129 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:58.567129 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:58.567129 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:58.567129 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:58.567129 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:54:58.567129 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:54:58.567129 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:54:58.567129 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:54:58.568022 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:58.568022 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:58.568022 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:58.568022 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:58.568022 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:58.568022 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:58.568022 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:58.568022 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:58.568022 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:58.568022 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:58.568022 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:58.568022 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:58.568022 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:58.568022 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:58.568022 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:58.568022 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:58.568022 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:54:58.569276 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:54:58.569276 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:58.569276 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:58.569276 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 437/861] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:54:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:58.575026 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d61ddf-7f08-47b0-aaa9-e6597d57106b None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:58.641988 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:58.677397 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d61ddf-7f08-47b0-aaa9-e6597d57106b None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:58.712414 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ebe665-299f-46d1-96d1-b1fdc30f2e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:58.713027 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ebe665-299f-46d1-96d1-b1fdc30f2e9d None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:58.713308 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ebe665-299f-46d1-96d1-b1fdc30f2e9d None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:58.714357 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ebe665-299f-46d1-96d1-b1fdc30f2e9d None admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:58.715197 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ebe665-299f-46d1-96d1-b1fdc30f2e9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:58.715502 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ebe665-299f-46d1-96d1-b1fdc30f2e9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:58.716883 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ebe665-299f-46d1-96d1-b1fdc30f2e9d None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:58.729298 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 425/862] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:54:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:58.735268 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c5247e-4648-4395-ba2b-a7c9ce7da68a None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:58.842468 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c5247e-4648-4395-ba2b-a7c9ce7da68a None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:58.878618 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553dcf1e-c981-402d-b2f0-541af0184858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:58.879277 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553dcf1e-c981-402d-b2f0-541af0184858 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:58.879557 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553dcf1e-c981-402d-b2f0-541af0184858 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:58.879828 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553dcf1e-c981-402d-b2f0-541af0184858 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:58.887926 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553dcf1e-c981-402d-b2f0-541af0184858 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:58.889363 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553dcf1e-c981-402d-b2f0-541af0184858 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:58.900490 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-553dcf1e-c981-402d-b2f0-541af0184858 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 18 16:54:58.900490 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:58.900490 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:58.900490 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:58.900490 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:58.900490 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:58.900490 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:58.900490 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:58.900490 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:58.900490 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:58.900490 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:58.900490 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:58.900490 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 16:54:58.900490 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 16:54:58.900490 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 16:54:58.900490 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 16:54:58.902330 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:58.902330 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:58.902330 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 16:54:58.902330 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:54:58.902330 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 16:54:58.902330 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:54:58.902330 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:58.902330 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:58.902330 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:58.902330 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:58.902330 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:58.902330 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:58.902330 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:58.902330 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:58.902330 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:58.902330 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:58.902330 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:58.902330 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:58.903991 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:58.903991 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:58.903991 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 16:54:58.903991 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 16:54:58.903991 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 16:54:58.903991 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 16:54:58.903991 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 16:54:58.903991 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 16:54:58.903991 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 18 16:54:58.903991 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:58.903991 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 438/863] 10.222.0.82 () {56 vars in 1117 bytes} [Thu Aug 18 16:54:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:58.908313 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ebe665-299f-46d1-96d1-b1fdc30f2e9d None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:59.031798 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ebe665-299f-46d1-96d1-b1fdc30f2e9d None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:59.059589 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:59.068018 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fae53c-9865-4aea-851c-e1505350440e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:59.068641 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fae53c-9865-4aea-851c-e1505350440e None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:59.068922 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fae53c-9865-4aea-851c-e1505350440e None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:59.069194 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fae53c-9865-4aea-851c-e1505350440e None admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:59.069921 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fae53c-9865-4aea-851c-e1505350440e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:59.070224 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fae53c-9865-4aea-851c-e1505350440e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:59.071700 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fae53c-9865-4aea-851c-e1505350440e None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:59.089965 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 426/864] 10.222.0.82 () {56 vars in 1149 bytes} [Thu Aug 18 16:54:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:59.096343 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553dcf1e-c981-402d-b2f0-541af0184858 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:59.127260 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:59.202245 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553dcf1e-c981-402d-b2f0-541af0184858 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:59.238256 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ce3afe-d6de-4a5c-9070-8fa2c183252c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:59.238867 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce3afe-d6de-4a5c-9070-8fa2c183252c None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:59.239174 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce3afe-d6de-4a5c-9070-8fa2c183252c None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:59.239445 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce3afe-d6de-4a5c-9070-8fa2c183252c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:59.313518 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ce3afe-d6de-4a5c-9070-8fa2c183252c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:59.314998 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ce3afe-d6de-4a5c-9070-8fa2c183252c None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:59.339210 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:59.407662 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c87030-6117-4968-8320-de8971545b13 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:59.408170 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c87030-6117-4968-8320-de8971545b13 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:59.408246 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c87030-6117-4968-8320-de8971545b13 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:59.409804 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 427/865] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:54:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:54:59.414887 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3c3822-8f8a-4b7a-9610-89b5425edc77 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:59.415201 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3c3822-8f8a-4b7a-9610-89b5425edc77 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:59.415481 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3c3822-8f8a-4b7a-9610-89b5425edc77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:59.427408 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 439/866] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:54:59 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 16:54:59.434252 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-8f3c3822-8f8a-4b7a-9610-89b5425edc77 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:54:59.435627 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-8f3c3822-8f8a-4b7a-9610-89b5425edc77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:54:59.436105 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ce3afe-d6de-4a5c-9070-8fa2c183252c None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:59.523285 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 428/867] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:54:59 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 16:54:59.549197 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ce3afe-d6de-4a5c-9070-8fa2c183252c None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:59.574061 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:59.585796 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7cee30-18d3-47e5-81e9-3b435ebfe65c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:59.586653 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7cee30-18d3-47e5-81e9-3b435ebfe65c None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:59.587955 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7cee30-18d3-47e5-81e9-3b435ebfe65c None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:59.587955 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7cee30-18d3-47e5-81e9-3b435ebfe65c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:59.598886 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7cee30-18d3-47e5-81e9-3b435ebfe65c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:59.600557 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7cee30-18d3-47e5-81e9-3b435ebfe65c None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:59.615855 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-bd7cee30-18d3-47e5-81e9-3b435ebfe65c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:59.615855 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:59.615855 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:59.615855 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:59.615855 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:59.615855 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:59.615855 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:59.615855 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:59.615855 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:59.615855 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:59.615855 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:59.615855 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:59.615855 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:54:59.615855 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:54:59.615855 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:54:59.615855 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:54:59.616797 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:59.616797 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:59.616797 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:59.616797 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:59.616797 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:59.616797 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:59.616797 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:59.616797 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:59.616797 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:59.616797 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:59.616797 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:59.616797 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:59.616797 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:59.616797 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:59.616797 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:59.616797 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:59.616797 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:54:59.617639 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:54:59.617639 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:54:59.617639 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:54:59.619462 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 440/868] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:54:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:54:59.626094 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3c3822-8f8a-4b7a-9610-89b5425edc77 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:59.734674 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3c3822-8f8a-4b7a-9610-89b5425edc77 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:59.750035 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:59.763749 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6201dc-c6ff-432f-a074-7b35ce8307ff None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:59.764175 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6201dc-c6ff-432f-a074-7b35ce8307ff None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:59.764562 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6201dc-c6ff-432f-a074-7b35ce8307ff None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:59.767634 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 441/869] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:54:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 16:54:59.771794 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999c716-2c83-4060-bbca-df2a1530fc8c None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:59.772196 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999c716-2c83-4060-bbca-df2a1530fc8c None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:59.776190 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999c716-2c83-4060-bbca-df2a1530fc8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:59.776312 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaba241-0a4b-4c27-8968-3a9db12b8059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:59.776406 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaba241-0a4b-4c27-8968-3a9db12b8059 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:59.776725 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaba241-0a4b-4c27-8968-3a9db12b8059 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:59.777046 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaba241-0a4b-4c27-8968-3a9db12b8059 None admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:59.777703 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaba241-0a4b-4c27-8968-3a9db12b8059 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:54:59.778007 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaba241-0a4b-4c27-8968-3a9db12b8059 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:59.786043 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaba241-0a4b-4c27-8968-3a9db12b8059 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:59.795537 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-9999c716-2c83-4060-bbca-df2a1530fc8c None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:54:59.796753 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-9999c716-2c83-4060-bbca-df2a1530fc8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:54:59.803766 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 429/870] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:54:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:54:59.809597 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaba241-0a4b-4c27-8968-3a9db12b8059 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:59.881927 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 442/871] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:54:59 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 16:54:59.882812 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9999c716-2c83-4060-bbca-df2a1530fc8c None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:54:59.929978 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaba241-0a4b-4c27-8968-3a9db12b8059 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:59.964989 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f448331b-04a4-4769-8d86-de68b8900d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:54:59.965962 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f448331b-04a4-4769-8d86-de68b8900d0c None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:59.966386 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f448331b-04a4-4769-8d86-de68b8900d0c None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:59.966801 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f448331b-04a4-4769-8d86-de68b8900d0c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:54:59.973720 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f448331b-04a4-4769-8d86-de68b8900d0c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:54:59.975414 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f448331b-04a4-4769-8d86-de68b8900d0c None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:54:59.986656 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-f448331b-04a4-4769-8d86-de68b8900d0c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:54:59.986656 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:54:59.986656 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:54:59.986656 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:54:59.986656 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:54:59.986656 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:54:59.986656 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:54:59.986656 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:54:59.986656 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:54:59.986656 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:54:59.986656 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:54:59.986656 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:54:59.986656 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 16:54:59.986656 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 16:54:59.986656 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 16:54:59.987709 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 16:54:59.987709 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:54:59.987709 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:54:59.987709 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:54:59.987709 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:54:59.987709 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:54:59.987709 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:54:59.987709 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:54:59.987709 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:54:59.987709 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:54:59.987709 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:54:59.987709 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:54:59.987709 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:54:59.987709 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:54:59.987709 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:54:59.987709 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:54:59.987709 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:54:59.988817 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 16:54:59.988817 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 16:54:59.988817 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 16:54:59.988817 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 16:54:59.988817 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 16:54:59.988817 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 16:54:59.988817 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:54:59.988817 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:54:59.989770 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9999c716-2c83-4060-bbca-df2a1530fc8c None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:54:59.992911 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:54:59.995718 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 430/872] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 16:54:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 16:54:59.996849 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c036ef8-165a-4978-99b0-0ea012899522 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:54:59.997143 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c036ef8-165a-4978-99b0-0ea012899522 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:54:59.997415 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c036ef8-165a-4978-99b0-0ea012899522 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:00.002766 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 431/873] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:54:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:55:00.002766 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c036ef8-165a-4978-99b0-0ea012899522 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:00.029078 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce1a378-e737-4145-a87f-cff2c8213d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:00.029716 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce1a378-e737-4145-a87f-cff2c8213d44 None admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:00.029907 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce1a378-e737-4145-a87f-cff2c8213d44 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:00.030184 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce1a378-e737-4145-a87f-cff2c8213d44 None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:00.030833 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce1a378-e737-4145-a87f-cff2c8213d44 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:00.032227 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce1a378-e737-4145-a87f-cff2c8213d44 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:00.038997 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:00.104559 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c036ef8-165a-4978-99b0-0ea012899522 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:00.109180 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 443/874] 10.222.0.82 () {60 vars in 1141 bytes} [Thu Aug 18 16:54:59 2022] POST /identity/v3/services => generated 269 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 16:55:00.116618 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f2819-e71f-48f1-bddf-4b15decf8c25 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:00.117241 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f2819-e71f-48f1-bddf-4b15decf8c25 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:00.117719 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f2819-e71f-48f1-bddf-4b15decf8c25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:00.145408 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-890f2819-e71f-48f1-bddf-4b15decf8c25 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:55:00.148080 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-890f2819-e71f-48f1-bddf-4b15decf8c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:55:00.159455 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b40ff27-acd5-4b56-a318-f9a263ff7dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:00.160086 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b40ff27-acd5-4b56-a318-f9a263ff7dbf None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:00.160364 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b40ff27-acd5-4b56-a318-f9a263ff7dbf None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:00.160632 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b40ff27-acd5-4b56-a318-f9a263ff7dbf None admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:00.161337 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b40ff27-acd5-4b56-a318-f9a263ff7dbf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:00.161640 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b40ff27-acd5-4b56-a318-f9a263ff7dbf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:00.163164 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b40ff27-acd5-4b56-a318-f9a263ff7dbf None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:00.176801 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 432/875] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 16:54:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:00.177468 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b40ff27-acd5-4b56-a318-f9a263ff7dbf None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:00.234953 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 444/876] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:55:00 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 16:55:00.234953 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890f2819-e71f-48f1-bddf-4b15decf8c25 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:00.286376 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:00.295761 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b40ff27-acd5-4b56-a318-f9a263ff7dbf None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:00.339840 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ce4196-5bb4-4aaf-b51a-01ddd6d9d526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:00.340750 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ce4196-5bb4-4aaf-b51a-01ddd6d9d526 None admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:00.341187 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ce4196-5bb4-4aaf-b51a-01ddd6d9d526 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:00.341597 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ce4196-5bb4-4aaf-b51a-01ddd6d9d526 None admin] PATH_INFO: `/v3/services` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:00.342381 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ce4196-5bb4-4aaf-b51a-01ddd6d9d526 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:00.343946 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ce4196-5bb4-4aaf-b51a-01ddd6d9d526 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:00.364931 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:00.371801 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890f2819-e71f-48f1-bddf-4b15decf8c25 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:00.405991 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afca6127-9e99-4ab9-a2c1-89753e6c8ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:00.406600 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca6127-9e99-4ab9-a2c1-89753e6c8ba9 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:00.406998 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca6127-9e99-4ab9-a2c1-89753e6c8ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:00.407236 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca6127-9e99-4ab9-a2c1-89753e6c8ba9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:00.407984 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afca6127-9e99-4ab9-a2c1-89753e6c8ba9 None admin] RBAC: Adding query filter params (role.id=5e0f402e1af84b44a723ee0730315d74, scope.project.id=2293c0aab61f44849d0d48b214cde7ea, user.id=5de19a54ed104bcdb35ee4e5e793f1e7) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:00.408293 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afca6127-9e99-4ab9-a2c1-89753e6c8ba9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:00.409721 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afca6127-9e99-4ab9-a2c1-89753e6c8ba9 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:00.415789 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 445/877] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 16:55:00 2022] GET /identity/v3/role_assignments?user.id=5de19a54ed104bcdb35ee4e5e793f1e7&scope.project.id=2293c0aab61f44849d0d48b214cde7ea&role.id=5e0f402e1af84b44a723ee0730315d74 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:00.482051 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 433/878] 10.222.0.82 () {60 vars in 1141 bytes} [Thu Aug 18 16:55:00 2022] POST /identity/v3/services => generated 251 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 16:55:00.516628 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afca6127-9e99-4ab9-a2c1-89753e6c8ba9 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:00.644663 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afca6127-9e99-4ab9-a2c1-89753e6c8ba9 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:00.688322 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7160c97-740d-476c-9dc6-112d0797fe3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:00.689194 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7160c97-740d-476c-9dc6-112d0797fe3c None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:00.689647 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7160c97-740d-476c-9dc6-112d0797fe3c None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:00.690062 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7160c97-740d-476c-9dc6-112d0797fe3c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:00.690801 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7160c97-740d-476c-9dc6-112d0797fe3c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:00.692407 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7160c97-740d-476c-9dc6-112d0797fe3c None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:00.703040 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-f7160c97-740d-476c-9dc6-112d0797fe3c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 16:55:00.703040 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:00.703040 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 16:55:00.703040 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 16:55:00.703040 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 16:55:00.703040 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 16:55:00.703040 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 16:55:00.703040 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 16:55:00.703040 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 16:55:00.703040 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:55:00.703040 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 16:55:00.703040 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:55:00.703040 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:00.703040 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:55:00.703040 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:55:00.703040 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:55:00.703040 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:55:00.704033 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:55:00.704033 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:55:00.704033 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:55:00.704033 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:55:00.704033 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:55:00.704033 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:55:00.704033 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 16:55:00.704033 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 16:55:00.704033 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 16:55:00.704033 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 16:55:00.704033 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:55:00.704033 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:55:00.704033 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:55:00.704033 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:55:00.704033 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:55:00.704033 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:55:00.705000 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:55:00.705000 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:55:00.705000 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:55:00.705000 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:55:00.705000 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:55:00.705000 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:55:00.705000 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:55:00.705000 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:55:00.705000 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:55:00.705000 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:55:00.705000 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 16:55:00.705000 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 16:55:00.705000 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 16:55:00.705000 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:55:00.705000 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 446/879] 10.222.0.82 () {56 vars in 1126 bytes} [Thu Aug 18 16:55:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:55:00.710384 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ce4196-5bb4-4aaf-b51a-01ddd6d9d526 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:00.834099 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ce4196-5bb4-4aaf-b51a-01ddd6d9d526 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:00.869766 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0a1878-bf34-4baf-8234-dc62ad2f8d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:00.870769 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0a1878-bf34-4baf-8234-dc62ad2f8d87 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:00.871221 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0a1878-bf34-4baf-8234-dc62ad2f8d87 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:00.871664 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0a1878-bf34-4baf-8234-dc62ad2f8d87 None admin] PATH_INFO: `/v3/services` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:00.872438 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0a1878-bf34-4baf-8234-dc62ad2f8d87 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:00.872886 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0a1878-bf34-4baf-8234-dc62ad2f8d87 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:00.874451 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0a1878-bf34-4baf-8234-dc62ad2f8d87 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:00.879914 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 434/880] 10.222.0.82 () {56 vars in 1122 bytes} [Thu Aug 18 16:55:00 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:55:00.889337 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7160c97-740d-476c-9dc6-112d0797fe3c None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:00.906314 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:00.987854 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7160c97-740d-476c-9dc6-112d0797fe3c None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:01.030141 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a6fadd-c099-4842-9df4-c4fb6f2c9d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:01.030816 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a6fadd-c099-4842-9df4-c4fb6f2c9d1a None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:01.031129 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a6fadd-c099-4842-9df4-c4fb6f2c9d1a None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:01.031402 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a6fadd-c099-4842-9df4-c4fb6f2c9d1a None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:01.032037 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a6fadd-c099-4842-9df4-c4fb6f2c9d1a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:01.032340 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a6fadd-c099-4842-9df4-c4fb6f2c9d1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:01.033712 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a6fadd-c099-4842-9df4-c4fb6f2c9d1a None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:01.043243 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 447/881] 10.222.0.82 () {56 vars in 1122 bytes} [Thu Aug 18 16:55:00 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:02.157702 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba29301-ced4-4d63-847f-fc6b6ecaf10e None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:02.157702 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba29301-ced4-4d63-847f-fc6b6ecaf10e None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:02.157702 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba29301-ced4-4d63-847f-fc6b6ecaf10e None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:02.157702 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 435/882] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:55:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:55:02.163285 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a86331f-bd91-4f26-bd2d-49cca54769d0 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:02.163678 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a86331f-bd91-4f26-bd2d-49cca54769d0 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:02.163859 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a86331f-bd91-4f26-bd2d-49cca54769d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:02.183996 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-0a86331f-bd91-4f26-bd2d-49cca54769d0 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:55:02.185253 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-0a86331f-bd91-4f26-bd2d-49cca54769d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:55:02.268590 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 448/883] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:55:02 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 16:55:02.560181 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d332c0-767a-49ca-93f5-8a6051cbf1ec None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:02.560567 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d332c0-767a-49ca-93f5-8a6051cbf1ec None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:02.560744 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d332c0-767a-49ca-93f5-8a6051cbf1ec None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:02.562304 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 436/884] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:55:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:55:02.571197 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94dcfaf-e7be-49f0-9cf8-b388d2369c23 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:02.571483 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94dcfaf-e7be-49f0-9cf8-b388d2369c23 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:02.571760 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94dcfaf-e7be-49f0-9cf8-b388d2369c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:02.592351 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-b94dcfaf-e7be-49f0-9cf8-b388d2369c23 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:55:02.593575 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-b94dcfaf-e7be-49f0-9cf8-b388d2369c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:55:02.601590 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:02.605784 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d332c0-767a-49ca-93f5-8a6051cbf1ec None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:02.625827 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:02.705343 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 449/885] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:55:02 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 16:55:02.706631 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a1c87-d711-4621-b795-1efb8a5cc6a3 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:02.706938 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a1c87-d711-4621-b795-1efb8a5cc6a3 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:02.707213 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a1c87-d711-4621-b795-1efb8a5cc6a3 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:02.708582 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 450/886] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:55:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:55:02.714098 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24709c8c-4d0e-4d3c-907c-590b1abc9bff None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:02.714444 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24709c8c-4d0e-4d3c-907c-590b1abc9bff None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:02.714656 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24709c8c-4d0e-4d3c-907c-590b1abc9bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:02.738806 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-24709c8c-4d0e-4d3c-907c-590b1abc9bff None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:55:02.739533 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d332c0-767a-49ca-93f5-8a6051cbf1ec None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:02.741990 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-24709c8c-4d0e-4d3c-907c-590b1abc9bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:55:02.787871 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1613b47d-763e-4ae4-bdbb-e10810bd16df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:02.788903 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1613b47d-763e-4ae4-bdbb-e10810bd16df None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:02.789404 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1613b47d-763e-4ae4-bdbb-e10810bd16df None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:02.789884 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1613b47d-763e-4ae4-bdbb-e10810bd16df None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:02.790655 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1613b47d-763e-4ae4-bdbb-e10810bd16df None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:02.793017 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1613b47d-763e-4ae4-bdbb-e10810bd16df None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:02.803067 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-1613b47d-763e-4ae4-bdbb-e10810bd16df None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 18 16:55:02.803067 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:02.803067 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 16:55:02.803067 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 16:55:02.803067 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 16:55:02.803067 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 16:55:02.803067 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 16:55:02.803067 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 16:55:02.803067 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 18 16:55:02.803067 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:55:02.803067 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 16:55:02.803067 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:55:02.803067 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:02.803067 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:55:02.803067 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:55:02.803067 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:55:02.803970 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:55:02.803970 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:55:02.803970 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:55:02.803970 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:55:02.803970 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:55:02.803970 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:55:02.803970 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:55:02.803970 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 16:55:02.803970 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 16:55:02.803970 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 16:55:02.803970 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 16:55:02.803970 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:55:02.803970 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:55:02.803970 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:55:02.803970 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:55:02.803970 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:55:02.803970 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:55:02.804836 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:55:02.804836 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:55:02.804836 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:55:02.804836 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:55:02.804836 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:55:02.804836 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:55:02.804836 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:55:02.804836 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:55:02.804836 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:55:02.804836 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:55:02.804836 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 16:55:02.804836 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 16:55:02.804836 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 18 16:55:02.804836 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:55:02.808986 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 437/887] 10.222.0.82 () {56 vars in 1132 bytes} [Thu Aug 18 16:55:02 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:55:02.814553 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1613b47d-763e-4ae4-bdbb-e10810bd16df None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:02.851045 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 451/888] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:55:02 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 16:55:02.904940 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:02.926658 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1613b47d-763e-4ae4-bdbb-e10810bd16df None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:02.962558 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969d4fb6-f326-45f5-8127-6618f07e754f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:02.964170 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969d4fb6-f326-45f5-8127-6618f07e754f None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:02.964509 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969d4fb6-f326-45f5-8127-6618f07e754f None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:02.964793 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969d4fb6-f326-45f5-8127-6618f07e754f None admin] PATH_INFO: `/v3/services` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:02.965423 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969d4fb6-f326-45f5-8127-6618f07e754f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:02.965788 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969d4fb6-f326-45f5-8127-6618f07e754f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:02.967204 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969d4fb6-f326-45f5-8127-6618f07e754f None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:02.972478 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 438/889] 10.222.0.82 () {56 vars in 1126 bytes} [Thu Aug 18 16:55:02 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:55:02.978101 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24709c8c-4d0e-4d3c-907c-590b1abc9bff None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:03.002339 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969d4fb6-f326-45f5-8127-6618f07e754f None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:03.054805 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:03.084697 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24709c8c-4d0e-4d3c-907c-590b1abc9bff None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:03.112958 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969d4fb6-f326-45f5-8127-6618f07e754f None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:03.120817 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45191254-c184-4339-b592-390ee563a0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:03.121429 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45191254-c184-4339-b592-390ee563a0bf None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:03.121711 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45191254-c184-4339-b592-390ee563a0bf None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:03.121990 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45191254-c184-4339-b592-390ee563a0bf None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:03.122616 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45191254-c184-4339-b592-390ee563a0bf None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:03.122945 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45191254-c184-4339-b592-390ee563a0bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:03.124356 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45191254-c184-4339-b592-390ee563a0bf None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:03.129104 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 452/890] 10.222.0.82 () {56 vars in 1126 bytes} [Thu Aug 18 16:55:02 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:03.130257 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c58b27-8733-4e67-bb8c-b37dc231ddb4 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:03.130542 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c58b27-8733-4e67-bb8c-b37dc231ddb4 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:03.130812 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c58b27-8733-4e67-bb8c-b37dc231ddb4 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:03.132194 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 453/891] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:55:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:55:03.138492 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa277b5b-9e41-4872-90de-3f8d27083756 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:03.138778 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa277b5b-9e41-4872-90de-3f8d27083756 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:03.139088 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa277b5b-9e41-4872-90de-3f8d27083756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:03.156780 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b15fef2-3a95-4583-8f0b-0fa6ecead94a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:03.157419 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b15fef2-3a95-4583-8f0b-0fa6ecead94a None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:03.157701 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b15fef2-3a95-4583-8f0b-0fa6ecead94a None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:03.157972 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b15fef2-3a95-4583-8f0b-0fa6ecead94a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:03.158495 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-aa277b5b-9e41-4872-90de-3f8d27083756 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:55:03.159359 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-aa277b5b-9e41-4872-90de-3f8d27083756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:55:03.169006 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b15fef2-3a95-4583-8f0b-0fa6ecead94a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:55:03.174102 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-7b15fef2-3a95-4583-8f0b-0fa6ecead94a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 18 16:55:03.174102 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:03.174102 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:55:03.174102 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:55:03.174102 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:55:03.174102 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:55:03.174102 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:55:03.174102 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:55:03.174102 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:55:03.174102 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:55:03.174102 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:55:03.174102 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:55:03.174102 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:55:03.174102 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:55:03.174102 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:55:03.174102 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:55:03.175043 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:55:03.175043 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:55:03.175043 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:55:03.175043 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:55:03.175043 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:55:03.175043 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:55:03.175043 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:55:03.175043 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:55:03.175043 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:55:03.175043 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:55:03.175043 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:55:03.175043 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:55:03.175043 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:55:03.175043 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:55:03.175043 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:55:03.175043 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:55:03.175967 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:55:03.175967 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:55:03.175967 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:55:03.175967 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:55:03.175967 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:55:03.175967 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:55:03.175967 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:55:03.175967 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:55:03.175967 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:55:03.175967 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:55:03.175967 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 18 16:55:03.175967 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:55:03.180109 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 439/892] 10.222.0.82 () {56 vars in 1120 bytes} [Thu Aug 18 16:55:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 16:55:03.180795 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b15fef2-3a95-4583-8f0b-0fa6ecead94a None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:03.184472 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:03.226312 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 454/893] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:55:03 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 16:55:03.227263 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa277b5b-9e41-4872-90de-3f8d27083756 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:03.286390 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b15fef2-3a95-4583-8f0b-0fa6ecead94a None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:03.307811 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:03.320140 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ad5c7c-8720-46da-844b-9c13f3e218b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:03.320748 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ad5c7c-8720-46da-844b-9c13f3e218b0 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:03.321037 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ad5c7c-8720-46da-844b-9c13f3e218b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:03.321309 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ad5c7c-8720-46da-844b-9c13f3e218b0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:03.322018 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ad5c7c-8720-46da-844b-9c13f3e218b0 None admin] RBAC: Adding query filter params (interface=public, service_id=8820c684d50f4b09b200670f3fd1b8f8, region_id=RegionOne) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:03.322343 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ad5c7c-8720-46da-844b-9c13f3e218b0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:03.324613 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ad5c7c-8720-46da-844b-9c13f3e218b0 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:03.336211 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 440/894] 10.222.0.82 () {56 vars in 1251 bytes} [Thu Aug 18 16:55:03 2022] GET /identity/v3/endpoints?service_id=8820c684d50f4b09b200670f3fd1b8f8&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:03.336211 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ad5c7c-8720-46da-844b-9c13f3e218b0 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:03.348914 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa277b5b-9e41-4872-90de-3f8d27083756 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:03.386545 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb47d37-130c-4d2a-8e52-8c9c985ea9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:03.387557 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb47d37-130c-4d2a-8e52-8c9c985ea9fa None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:03.387999 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb47d37-130c-4d2a-8e52-8c9c985ea9fa None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:03.388273 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb47d37-130c-4d2a-8e52-8c9c985ea9fa None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:03.388852 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb47d37-130c-4d2a-8e52-8c9c985ea9fa None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:03.390443 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb47d37-130c-4d2a-8e52-8c9c985ea9fa None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:03.399452 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:03.400560 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-5fb47d37-130c-4d2a-8e52-8c9c985ea9fa None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 18 16:55:03.400560 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:03.400560 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 16:55:03.400560 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 16:55:03.400560 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 16:55:03.400560 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 16:55:03.400560 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 16:55:03.400560 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 16:55:03.400560 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 18 16:55:03.400560 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:55:03.400560 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 16:55:03.400560 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:55:03.400560 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:03.400560 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:55:03.400560 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:55:03.400560 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:55:03.400560 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:55:03.401602 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:55:03.401602 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:55:03.401602 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:55:03.401602 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:55:03.401602 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:55:03.401602 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:55:03.401602 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 16:55:03.401602 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 16:55:03.401602 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 16:55:03.401602 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 16:55:03.401602 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:55:03.401602 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:55:03.401602 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:55:03.401602 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:55:03.401602 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:55:03.401602 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:55:03.402411 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:55:03.402411 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:55:03.402411 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:55:03.402411 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:55:03.402411 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:55:03.402411 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:55:03.402411 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:55:03.402411 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:55:03.402411 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:55:03.402411 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:55:03.402411 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 16:55:03.402411 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 16:55:03.402411 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 18 16:55:03.402411 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:55:03.402411 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 455/895] 10.222.0.82 () {56 vars in 1114 bytes} [Thu Aug 18 16:55:03 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:55:03.403664 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb47d37-130c-4d2a-8e52-8c9c985ea9fa None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:03.440417 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ad5c7c-8720-46da-844b-9c13f3e218b0 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:03.475316 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d941c2e-3d1c-4b24-9ba4-685766cc1f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:03.475918 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d941c2e-3d1c-4b24-9ba4-685766cc1f32 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:03.476197 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d941c2e-3d1c-4b24-9ba4-685766cc1f32 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:03.476468 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d941c2e-3d1c-4b24-9ba4-685766cc1f32 None admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:03.477115 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d941c2e-3d1c-4b24-9ba4-685766cc1f32 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:03.477415 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d941c2e-3d1c-4b24-9ba4-685766cc1f32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:03.478812 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d941c2e-3d1c-4b24-9ba4-685766cc1f32 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:03.492112 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 441/896] 10.222.0.82 () {56 vars in 1115 bytes} [Thu Aug 18 16:55:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:03.492112 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d941c2e-3d1c-4b24-9ba4-685766cc1f32 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:03.519284 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb47d37-130c-4d2a-8e52-8c9c985ea9fa None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:03.547741 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:03.553132 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b2ad71-d21c-406d-a014-34653e70ff70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:03.553735 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2ad71-d21c-406d-a014-34653e70ff70 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:03.554030 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2ad71-d21c-406d-a014-34653e70ff70 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:03.554303 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2ad71-d21c-406d-a014-34653e70ff70 None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:03.554918 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b2ad71-d21c-406d-a014-34653e70ff70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:03.556291 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b2ad71-d21c-406d-a014-34653e70ff70 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:03.562309 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 456/897] 10.222.0.82 () {56 vars in 1087 bytes} [Thu Aug 18 16:55:03 2022] GET /identity/v3/services => generated 1268 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 16:55:03.563119 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b2ad71-d21c-406d-a014-34653e70ff70 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:03.592877 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d941c2e-3d1c-4b24-9ba4-685766cc1f32 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:03.624205 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:03.632878 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418a1a21-d9a3-4362-98bf-6796ac8b0c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:03.633495 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a1a21-d9a3-4362-98bf-6796ac8b0c19 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:03.638555 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a1a21-d9a3-4362-98bf-6796ac8b0c19 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:03.638555 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a1a21-d9a3-4362-98bf-6796ac8b0c19 None admin] PATH_INFO: `/v3/services` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:03.638555 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418a1a21-d9a3-4362-98bf-6796ac8b0c19 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:03.638555 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418a1a21-d9a3-4362-98bf-6796ac8b0c19 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:03.638555 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418a1a21-d9a3-4362-98bf-6796ac8b0c19 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:03.642330 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 442/898] 10.222.0.82 () {56 vars in 1114 bytes} [Thu Aug 18 16:55:03 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:03.643438 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418a1a21-d9a3-4362-98bf-6796ac8b0c19 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:03.694255 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b2ad71-d21c-406d-a014-34653e70ff70 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:03.734958 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedd3ceb-931b-40e6-bc1f-d7869557c9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:03.735602 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd3ceb-931b-40e6-bc1f-d7869557c9f1 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:03.735887 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd3ceb-931b-40e6-bc1f-d7869557c9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:03.736163 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd3ceb-931b-40e6-bc1f-d7869557c9f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:03.748896 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedd3ceb-931b-40e6-bc1f-d7869557c9f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:03.750426 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedd3ceb-931b-40e6-bc1f-d7869557c9f1 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:03.759770 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418a1a21-d9a3-4362-98bf-6796ac8b0c19 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:03.764837 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-aedd3ceb-931b-40e6-bc1f-d7869557c9f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:55:03.764837 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:03.764837 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:55:03.764837 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:55:03.764837 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:55:03.764837 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:55:03.764837 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:55:03.764837 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:55:03.764837 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:55:03.764837 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:55:03.764837 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:55:03.764837 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:55:03.764837 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:55:03.764837 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:55:03.764837 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:55:03.764837 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:55:03.765759 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:55:03.765759 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:55:03.765759 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:55:03.765759 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:55:03.765759 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:55:03.765759 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:55:03.765759 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:55:03.765759 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:55:03.765759 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:55:03.765759 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:55:03.765759 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:55:03.765759 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:55:03.765759 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:55:03.765759 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:55:03.765759 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:55:03.765759 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:55:03.765759 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:55:03.766647 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:55:03.766647 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:55:03.766647 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:55:03.766647 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 457/899] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:55:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:55:03.771420 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedd3ceb-931b-40e6-bc1f-d7869557c9f1 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:03.801657 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4549d6-2b6e-414f-a50d-188cbd82d7fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:03.802269 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4549d6-2b6e-414f-a50d-188cbd82d7fd None admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:03.802553 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4549d6-2b6e-414f-a50d-188cbd82d7fd None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:03.802831 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4549d6-2b6e-414f-a50d-188cbd82d7fd None admin] PATH_INFO: `/v3/services` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:03.803494 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4549d6-2b6e-414f-a50d-188cbd82d7fd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:03.804876 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4549d6-2b6e-414f-a50d-188cbd82d7fd None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:03.825717 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:03.877584 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedd3ceb-931b-40e6-bc1f-d7869557c9f1 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:03.886933 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 443/900] 10.222.0.82 () {60 vars in 1141 bytes} [Thu Aug 18 16:55:03 2022] POST /identity/v3/services => generated 241 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 16:55:03.887845 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4549d6-2b6e-414f-a50d-188cbd82d7fd None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:03.943456 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c1b2ac-fb5e-4158-8707-b61adb4cc1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:03.945068 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c1b2ac-fb5e-4158-8707-b61adb4cc1de None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:03.945787 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c1b2ac-fb5e-4158-8707-b61adb4cc1de None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:03.946210 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c1b2ac-fb5e-4158-8707-b61adb4cc1de None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:03.947301 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c1b2ac-fb5e-4158-8707-b61adb4cc1de None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:03.948594 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c1b2ac-fb5e-4158-8707-b61adb4cc1de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:03.950801 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c1b2ac-fb5e-4158-8707-b61adb4cc1de None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:03.953437 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:03.959986 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 458/901] 10.222.0.82 () {56 vars in 1114 bytes} [Thu Aug 18 16:55:03 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:03.966139 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c1b2ac-fb5e-4158-8707-b61adb4cc1de None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:04.017738 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4549d6-2b6e-414f-a50d-188cbd82d7fd None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:04.058141 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7722b12-7ba0-47f6-bcec-59b6ba68f10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:04.058766 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7722b12-7ba0-47f6-bcec-59b6ba68f10a None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:04.059658 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7722b12-7ba0-47f6-bcec-59b6ba68f10a None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:04.059658 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7722b12-7ba0-47f6-bcec-59b6ba68f10a None admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:04.060686 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7722b12-7ba0-47f6-bcec-59b6ba68f10a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:04.060686 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7722b12-7ba0-47f6-bcec-59b6ba68f10a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:04.061731 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7722b12-7ba0-47f6-bcec-59b6ba68f10a None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:04.075180 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 444/902] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:55:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:04.083270 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7722b12-7ba0-47f6-bcec-59b6ba68f10a None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:04.102704 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c1b2ac-fb5e-4158-8707-b61adb4cc1de None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:04.110923 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:04.138197 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29394d98-de88-49d5-8889-4aa5b49ff43e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:04.139054 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29394d98-de88-49d5-8889-4aa5b49ff43e None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:04.139495 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29394d98-de88-49d5-8889-4aa5b49ff43e None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:04.139907 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29394d98-de88-49d5-8889-4aa5b49ff43e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:04.140751 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29394d98-de88-49d5-8889-4aa5b49ff43e None admin] RBAC: Adding query filter params (interface=public, service_id=dcf576ef6290401ab552491604b8ae8b, region_id=RegionOne) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:04.141324 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29394d98-de88-49d5-8889-4aa5b49ff43e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:04.142817 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29394d98-de88-49d5-8889-4aa5b49ff43e None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:04.148210 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 459/903] 10.222.0.82 () {56 vars in 1251 bytes} [Thu Aug 18 16:55:03 2022] GET /identity/v3/endpoints?service_id=dcf576ef6290401ab552491604b8ae8b&interface=public®ion_id=RegionOne => generated 194 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:04.154320 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29394d98-de88-49d5-8889-4aa5b49ff43e None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:04.189447 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7722b12-7ba0-47f6-bcec-59b6ba68f10a None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:04.203645 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:04.225507 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5690c802-1611-4a14-8a14-5cc2e47cdfca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:04.226247 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5690c802-1611-4a14-8a14-5cc2e47cdfca None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:04.226607 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5690c802-1611-4a14-8a14-5cc2e47cdfca None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:04.226980 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5690c802-1611-4a14-8a14-5cc2e47cdfca None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:04.234724 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5690c802-1611-4a14-8a14-5cc2e47cdfca None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:04.236294 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5690c802-1611-4a14-8a14-5cc2e47cdfca None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:04.249012 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-5690c802-1611-4a14-8a14-5cc2e47cdfca None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 18 16:55:04.249012 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:04.249012 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:55:04.249012 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:55:04.249012 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:55:04.249012 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:55:04.249012 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:55:04.249012 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:55:04.249012 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:55:04.249012 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:55:04.249012 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:55:04.249012 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:55:04.249012 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 16:55:04.249012 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 16:55:04.249012 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 16:55:04.249012 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 16:55:04.250124 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:55:04.250124 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:55:04.250124 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 16:55:04.250124 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:55:04.250124 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 16:55:04.250124 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:55:04.250124 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:55:04.250124 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:55:04.250124 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:55:04.250124 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:55:04.250124 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:55:04.250124 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:55:04.250124 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:55:04.250124 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:55:04.250124 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:55:04.250124 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:55:04.250124 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:55:04.250124 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:55:04.251253 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:55:04.251253 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:55:04.251253 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 16:55:04.251253 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 16:55:04.251253 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 16:55:04.251253 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 16:55:04.251253 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 16:55:04.251253 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 16:55:04.251253 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 18 16:55:04.251253 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:55:04.257826 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 445/904] 10.222.0.82 () {56 vars in 1117 bytes} [Thu Aug 18 16:55:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 16:55:04.258819 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5690c802-1611-4a14-8a14-5cc2e47cdfca None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:04.273606 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29394d98-de88-49d5-8889-4aa5b49ff43e None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:04.323939 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63902ed3-be06-4842-a9c9-3efbdea64eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:04.324991 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63902ed3-be06-4842-a9c9-3efbdea64eb9 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:04.325496 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63902ed3-be06-4842-a9c9-3efbdea64eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:04.325964 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63902ed3-be06-4842-a9c9-3efbdea64eb9 None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:04.326753 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63902ed3-be06-4842-a9c9-3efbdea64eb9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:04.328539 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63902ed3-be06-4842-a9c9-3efbdea64eb9 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:04.341009 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 460/905] 10.222.0.82 () {56 vars in 1087 bytes} [Thu Aug 18 16:55:04 2022] GET /identity/v3/services => generated 1497 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 16:55:04.378389 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5690c802-1611-4a14-8a14-5cc2e47cdfca None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:04.417035 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc73d4a-d66b-4656-ae5f-5bdbc721f918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:04.417590 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc73d4a-d66b-4656-ae5f-5bdbc721f918 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:04.417875 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc73d4a-d66b-4656-ae5f-5bdbc721f918 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:04.418146 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc73d4a-d66b-4656-ae5f-5bdbc721f918 None admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:04.418888 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc73d4a-d66b-4656-ae5f-5bdbc721f918 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:04.419225 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc73d4a-d66b-4656-ae5f-5bdbc721f918 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:04.420696 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc73d4a-d66b-4656-ae5f-5bdbc721f918 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:04.441512 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 446/906] 10.222.0.82 () {56 vars in 1149 bytes} [Thu Aug 18 16:55:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:55:04.447305 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63902ed3-be06-4842-a9c9-3efbdea64eb9 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:04.486873 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:04.559893 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63902ed3-be06-4842-a9c9-3efbdea64eb9 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:04.594665 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9856589-1a6f-48b6-a587-7569746bdd6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:04.595304 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9856589-1a6f-48b6-a587-7569746bdd6a None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:04.595584 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9856589-1a6f-48b6-a587-7569746bdd6a None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:04.595853 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9856589-1a6f-48b6-a587-7569746bdd6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:04.664063 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9856589-1a6f-48b6-a587-7569746bdd6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:04.665563 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9856589-1a6f-48b6-a587-7569746bdd6a None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:04.711418 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:04.773623 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 461/907] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:55:04 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 16:55:04.779689 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc73d4a-d66b-4656-ae5f-5bdbc721f918 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:04.816831 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:04.894726 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc73d4a-d66b-4656-ae5f-5bdbc721f918 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:04.928662 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e0ccde-3358-46a4-a2ac-d4ac7ff10d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:04.929300 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e0ccde-3358-46a4-a2ac-d4ac7ff10d85 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:04.929578 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e0ccde-3358-46a4-a2ac-d4ac7ff10d85 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:04.929854 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e0ccde-3358-46a4-a2ac-d4ac7ff10d85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:04.940905 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e0ccde-3358-46a4-a2ac-d4ac7ff10d85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:04.942462 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e0ccde-3358-46a4-a2ac-d4ac7ff10d85 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:04.958791 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-c1e0ccde-3358-46a4-a2ac-d4ac7ff10d85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:55:04.958791 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:04.958791 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:55:04.958791 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:55:04.958791 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:55:04.958791 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:55:04.958791 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:55:04.958791 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:55:04.958791 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:55:04.958791 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:55:04.958791 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:55:04.958791 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:55:04.958791 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:55:04.958791 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:55:04.958791 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:55:04.958791 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:55:04.959818 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:55:04.959818 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:55:04.959818 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:55:04.959818 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:55:04.959818 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:55:04.959818 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:55:04.959818 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:55:04.959818 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:55:04.959818 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:55:04.959818 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:55:04.959818 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:55:04.959818 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:55:04.959818 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:55:04.959818 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:55:04.959818 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:55:04.959818 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:55:04.959818 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:55:04.960655 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:55:04.960655 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:55:04.960655 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:55:04.960655 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 447/908] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:55:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:55:04.970074 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9856589-1a6f-48b6-a587-7569746bdd6a None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:05.068341 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9856589-1a6f-48b6-a587-7569746bdd6a None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:05.107174 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00390ec5-b4da-4e70-830b-c9dba9738416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:05.107792 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00390ec5-b4da-4e70-830b-c9dba9738416 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:05.108075 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00390ec5-b4da-4e70-830b-c9dba9738416 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:05.108347 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00390ec5-b4da-4e70-830b-c9dba9738416 None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:05.108989 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00390ec5-b4da-4e70-830b-c9dba9738416 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:05.109289 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00390ec5-b4da-4e70-830b-c9dba9738416 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:05.110660 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00390ec5-b4da-4e70-830b-c9dba9738416 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:05.119273 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:05.123569 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 462/909] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:55:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:05.132000 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e0ccde-3358-46a4-a2ac-d4ac7ff10d85 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:05.216627 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:05.243324 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e0ccde-3358-46a4-a2ac-d4ac7ff10d85 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:05.270443 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b5e8db-3fbb-409a-bf83-086005415964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:05.271159 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b5e8db-3fbb-409a-bf83-086005415964 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:05.271439 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b5e8db-3fbb-409a-bf83-086005415964 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:05.271749 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b5e8db-3fbb-409a-bf83-086005415964 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:05.278471 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b5e8db-3fbb-409a-bf83-086005415964 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:05.279935 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b5e8db-3fbb-409a-bf83-086005415964 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:05.290873 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-96b5e8db-3fbb-409a-bf83-086005415964 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:55:05.290873 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:05.290873 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:55:05.290873 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:55:05.290873 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:55:05.290873 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:55:05.290873 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:55:05.290873 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:55:05.290873 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:55:05.290873 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:55:05.290873 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:55:05.290873 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:55:05.290873 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 16:55:05.290873 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 16:55:05.290873 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 16:55:05.292265 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 16:55:05.292265 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:55:05.292265 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:55:05.292265 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:55:05.292265 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:55:05.292265 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:55:05.292265 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:55:05.292265 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:55:05.292265 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:55:05.292265 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:55:05.292265 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:55:05.292265 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:55:05.292265 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:55:05.292265 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:55:05.292265 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:55:05.292265 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:55:05.292265 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:55:05.293537 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 16:55:05.293537 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 16:55:05.293537 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 16:55:05.293537 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 16:55:05.293537 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 16:55:05.293537 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 16:55:05.293537 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:55:05.293537 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:55:05.293537 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 448/910] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 16:55:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:55:05.298637 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00390ec5-b4da-4e70-830b-c9dba9738416 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:05.406120 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00390ec5-b4da-4e70-830b-c9dba9738416 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:05.441158 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c6b77e-9b9f-4114-82db-5fcecb218834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:05.442019 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c6b77e-9b9f-4114-82db-5fcecb218834 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:05.442437 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c6b77e-9b9f-4114-82db-5fcecb218834 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:05.442849 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c6b77e-9b9f-4114-82db-5fcecb218834 None admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:05.443690 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c6b77e-9b9f-4114-82db-5fcecb218834 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:05.444168 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c6b77e-9b9f-4114-82db-5fcecb218834 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:05.445776 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c6b77e-9b9f-4114-82db-5fcecb218834 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:05.458624 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 463/911] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 16:55:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:05.465714 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b5e8db-3fbb-409a-bf83-086005415964 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:05.570490 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b5e8db-3fbb-409a-bf83-086005415964 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:05.576906 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97586e26-c526-4458-b2f2-533351103e80 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:05.577541 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97586e26-c526-4458-b2f2-533351103e80 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:05.578015 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97586e26-c526-4458-b2f2-533351103e80 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:05.579983 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 464/912] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:55:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 16:55:05.587885 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505bbe62-f404-4a0c-9f96-f49b0f04d9b0 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:05.590068 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505bbe62-f404-4a0c-9f96-f49b0f04d9b0 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:05.590570 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505bbe62-f404-4a0c-9f96-f49b0f04d9b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:05.611831 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-505bbe62-f404-4a0c-9f96-f49b0f04d9b0 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:55:05.612848 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04612f7-eec0-4271-accd-d83e5886baf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:05.613467 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04612f7-eec0-4271-accd-d83e5886baf8 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:05.613750 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04612f7-eec0-4271-accd-d83e5886baf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:05.614023 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04612f7-eec0-4271-accd-d83e5886baf8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:05.614303 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-505bbe62-f404-4a0c-9f96-f49b0f04d9b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:55:05.614753 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04612f7-eec0-4271-accd-d83e5886baf8 None admin] RBAC: Adding query filter params (role.id=bae4160aa9de4d69b42641fd18c53e55, scope.project.id=2293c0aab61f44849d0d48b214cde7ea, user.id=5de19a54ed104bcdb35ee4e5e793f1e7) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:05.615095 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04612f7-eec0-4271-accd-d83e5886baf8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:05.616544 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04612f7-eec0-4271-accd-d83e5886baf8 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:05.626968 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 449/913] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 16:55:05 2022] GET /identity/v3/role_assignments?user.id=5de19a54ed104bcdb35ee4e5e793f1e7&scope.project.id=2293c0aab61f44849d0d48b214cde7ea&role.id=bae4160aa9de4d69b42641fd18c53e55 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:05.677803 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:05.698936 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 465/914] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:55:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 16:55:05.926793 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6554be-5e18-4bb0-b722-55ce2df93864 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:05.927047 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6554be-5e18-4bb0-b722-55ce2df93864 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:05.927287 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6554be-5e18-4bb0-b722-55ce2df93864 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:05.928908 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 450/915] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:55:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:55:05.934849 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c3b4a-7922-4331-b4c0-dd9efd2ee3ee None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:05.935167 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c3b4a-7922-4331-b4c0-dd9efd2ee3ee None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:05.935443 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c3b4a-7922-4331-b4c0-dd9efd2ee3ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:05.954132 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-7d3c3b4a-7922-4331-b4c0-dd9efd2ee3ee None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:55:05.955352 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-7d3c3b4a-7922-4331-b4c0-dd9efd2ee3ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:55:05.982324 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6554be-5e18-4bb0-b722-55ce2df93864 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:05.992126 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:06.027023 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 466/916] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:55:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 16:55:06.107597 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6554be-5e18-4bb0-b722-55ce2df93864 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:06.143607 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d315a35-5bd2-4ea9-8ea0-ceca0bda65a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:06.144518 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d315a35-5bd2-4ea9-8ea0-ceca0bda65a5 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:06.144949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d315a35-5bd2-4ea9-8ea0-ceca0bda65a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:06.145382 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d315a35-5bd2-4ea9-8ea0-ceca0bda65a5 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:06.146103 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d315a35-5bd2-4ea9-8ea0-ceca0bda65a5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:06.147649 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d315a35-5bd2-4ea9-8ea0-ceca0bda65a5 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:06.157563 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-3d315a35-5bd2-4ea9-8ea0-ceca0bda65a5 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 18 16:55:06.157563 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:06.157563 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 16:55:06.157563 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 16:55:06.157563 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 16:55:06.157563 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 16:55:06.157563 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 16:55:06.157563 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 16:55:06.157563 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 18 16:55:06.157563 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:55:06.157563 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 16:55:06.157563 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:55:06.157563 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:06.157563 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:55:06.157563 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:55:06.157563 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:55:06.158706 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:55:06.158706 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:55:06.158706 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:55:06.158706 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:55:06.158706 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:55:06.158706 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:55:06.158706 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:55:06.158706 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 16:55:06.158706 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 16:55:06.158706 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 16:55:06.158706 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 16:55:06.158706 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:55:06.158706 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:55:06.158706 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:55:06.158706 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:55:06.158706 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:55:06.158706 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:55:06.159807 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:55:06.159807 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:55:06.159807 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:55:06.159807 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:55:06.159807 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:55:06.159807 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:55:06.159807 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:55:06.159807 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:55:06.159807 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:55:06.159807 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:55:06.159807 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 16:55:06.159807 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 16:55:06.159807 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 18 16:55:06.159807 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:55:06.162210 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 451/917] 10.222.0.82 () {56 vars in 1132 bytes} [Thu Aug 18 16:55:05 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:55:06.168007 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3c3b4a-7922-4331-b4c0-dd9efd2ee3ee None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:06.273701 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3c3b4a-7922-4331-b4c0-dd9efd2ee3ee None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:06.288870 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:06.305863 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d315a35-5bd2-4ea9-8ea0-ceca0bda65a5 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:06.310114 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63192e3-4855-4380-bb2b-ea063238b743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:06.310740 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63192e3-4855-4380-bb2b-ea063238b743 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:06.311898 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63192e3-4855-4380-bb2b-ea063238b743 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:06.312183 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63192e3-4855-4380-bb2b-ea063238b743 None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:06.312823 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63192e3-4855-4380-bb2b-ea063238b743 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:06.313130 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63192e3-4855-4380-bb2b-ea063238b743 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:06.314513 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63192e3-4855-4380-bb2b-ea063238b743 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:06.321117 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 467/918] 10.222.0.82 () {56 vars in 1126 bytes} [Thu Aug 18 16:55:06 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:06.327140 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63192e3-4855-4380-bb2b-ea063238b743 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:06.386858 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:06.424875 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d315a35-5bd2-4ea9-8ea0-ceca0bda65a5 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:06.450141 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63192e3-4855-4380-bb2b-ea063238b743 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:06.460262 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f5fda4-2078-4017-b27c-b33c7c93e490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:06.461164 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f5fda4-2078-4017-b27c-b33c7c93e490 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:06.461661 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f5fda4-2078-4017-b27c-b33c7c93e490 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:06.462150 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f5fda4-2078-4017-b27c-b33c7c93e490 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:06.463700 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f5fda4-2078-4017-b27c-b33c7c93e490 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:06.465393 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f5fda4-2078-4017-b27c-b33c7c93e490 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:06.474749 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-b8f5fda4-2078-4017-b27c-b33c7c93e490 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 16:55:06.474749 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:06.474749 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 16:55:06.474749 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 16:55:06.474749 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 16:55:06.474749 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 16:55:06.474749 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 16:55:06.474749 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 16:55:06.474749 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 16:55:06.474749 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:55:06.474749 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 16:55:06.474749 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:55:06.474749 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:06.474749 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:55:06.474749 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:55:06.474749 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:55:06.474749 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:55:06.475719 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:55:06.475719 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:55:06.475719 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:55:06.475719 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:55:06.475719 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:55:06.475719 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:55:06.475719 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 16:55:06.475719 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 16:55:06.475719 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 16:55:06.475719 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 16:55:06.475719 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:55:06.475719 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:55:06.475719 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:55:06.475719 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:55:06.475719 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:55:06.475719 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:55:06.476548 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:55:06.476548 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:55:06.476548 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:55:06.476548 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:55:06.476548 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:55:06.476548 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:55:06.476548 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:55:06.476548 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:55:06.476548 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:55:06.476548 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:55:06.476548 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 16:55:06.476548 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 16:55:06.476548 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 16:55:06.476548 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:55:06.479878 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 452/919] 10.222.0.82 () {56 vars in 1126 bytes} [Thu Aug 18 16:55:06 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:55:06.486008 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f5fda4-2078-4017-b27c-b33c7c93e490 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:06.490139 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c4f7ad-843a-4b99-8876-43fee1580f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:06.491164 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4f7ad-843a-4b99-8876-43fee1580f8d None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:06.492406 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4f7ad-843a-4b99-8876-43fee1580f8d None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:06.492897 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4f7ad-843a-4b99-8876-43fee1580f8d None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:06.493744 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c4f7ad-843a-4b99-8876-43fee1580f8d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:06.494777 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c4f7ad-843a-4b99-8876-43fee1580f8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:06.496452 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c4f7ad-843a-4b99-8876-43fee1580f8d None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:06.508087 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 468/920] 10.222.0.82 () {56 vars in 1126 bytes} [Thu Aug 18 16:55:06 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:55:06.509685 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c4f7ad-843a-4b99-8876-43fee1580f8d None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:06.564240 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:06.599052 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f5fda4-2078-4017-b27c-b33c7c93e490 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:06.623554 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c4f7ad-843a-4b99-8876-43fee1580f8d None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:06.635339 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d6163a-e76a-40e2-acee-39066d09983e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:06.635954 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d6163a-e76a-40e2-acee-39066d09983e None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:06.636229 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d6163a-e76a-40e2-acee-39066d09983e None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:06.636501 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d6163a-e76a-40e2-acee-39066d09983e None admin] PATH_INFO: `/v3/services` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:06.637119 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d6163a-e76a-40e2-acee-39066d09983e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:06.637418 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d6163a-e76a-40e2-acee-39066d09983e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:06.638791 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d6163a-e76a-40e2-acee-39066d09983e None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:06.639917 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:06.644301 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 453/921] 10.222.0.82 () {56 vars in 1122 bytes} [Thu Aug 18 16:55:06 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:06.649300 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6effd168-e06e-4b97-8800-fcf9a6c36921 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:06.649300 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6effd168-e06e-4b97-8800-fcf9a6c36921 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:06.649300 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6effd168-e06e-4b97-8800-fcf9a6c36921 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:06.649300 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 454/922] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:55:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:55:06.654153 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6effd168-e06e-4b97-8800-fcf9a6c36921 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:06.667389 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4021ddf0-db14-498a-bf0a-65844bed0a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:06.668149 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4021ddf0-db14-498a-bf0a-65844bed0a04 None admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:06.668451 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4021ddf0-db14-498a-bf0a-65844bed0a04 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:06.668710 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4021ddf0-db14-498a-bf0a-65844bed0a04 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:06.669487 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4021ddf0-db14-498a-bf0a-65844bed0a04 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:06.670873 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4021ddf0-db14-498a-bf0a-65844bed0a04 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:06.750206 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 469/923] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 16:55:06 2022] POST /identity/v3/endpoints => generated 347 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 16:55:06.751460 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afdc8d3-f9df-4974-a9ba-a74d8507dd17 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:06.751755 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afdc8d3-f9df-4974-a9ba-a74d8507dd17 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:06.752037 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afdc8d3-f9df-4974-a9ba-a74d8507dd17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:06.763799 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6effd168-e06e-4b97-8800-fcf9a6c36921 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:06.774049 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-1afdc8d3-f9df-4974-a9ba-a74d8507dd17 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:55:06.776423 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-1afdc8d3-f9df-4974-a9ba-a74d8507dd17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:55:06.816807 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f36a55c-cc1e-4d94-890e-9949901ed22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:06.817562 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f36a55c-cc1e-4d94-890e-9949901ed22c None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:06.817838 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f36a55c-cc1e-4d94-890e-9949901ed22c None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:06.818104 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f36a55c-cc1e-4d94-890e-9949901ed22c None admin] PATH_INFO: `/v3/services` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:06.818737 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f36a55c-cc1e-4d94-890e-9949901ed22c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:06.819083 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f36a55c-cc1e-4d94-890e-9949901ed22c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:06.820479 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f36a55c-cc1e-4d94-890e-9949901ed22c None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:06.832205 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 455/924] 10.222.0.82 () {56 vars in 1122 bytes} [Thu Aug 18 16:55:06 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:55:06.833829 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f36a55c-cc1e-4d94-890e-9949901ed22c None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:06.875087 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 470/925] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:55:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 16:55:06.909211 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:06.953194 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f36a55c-cc1e-4d94-890e-9949901ed22c None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:06.988550 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4b1ea1-571d-4dd1-96be-2e843ced28e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:06.989154 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b1ea1-571d-4dd1-96be-2e843ced28e1 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:06.989432 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b1ea1-571d-4dd1-96be-2e843ced28e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:06.989700 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b1ea1-571d-4dd1-96be-2e843ced28e1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:06.990386 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4b1ea1-571d-4dd1-96be-2e843ced28e1 None admin] RBAC: Adding query filter params (interface=public, service_id=6d5ec5c4407e40758cd24a0aa6d8507f, region_id=RegionOne) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:06.990691 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4b1ea1-571d-4dd1-96be-2e843ced28e1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:06.992108 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4b1ea1-571d-4dd1-96be-2e843ced28e1 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:06.998133 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 456/926] 10.222.0.82 () {56 vars in 1251 bytes} [Thu Aug 18 16:55:06 2022] GET /identity/v3/endpoints?service_id=6d5ec5c4407e40758cd24a0aa6d8507f&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:07.003719 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afdc8d3-f9df-4974-a9ba-a74d8507dd17 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:07.034735 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:07.109932 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afdc8d3-f9df-4974-a9ba-a74d8507dd17 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:07.146782 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbc9628-8848-4d26-a815-ac11180adb5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:07.147655 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc9628-8848-4d26-a815-ac11180adb5c None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:07.148103 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc9628-8848-4d26-a815-ac11180adb5c None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:07.148521 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc9628-8848-4d26-a815-ac11180adb5c None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:07.149275 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbc9628-8848-4d26-a815-ac11180adb5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:07.151451 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbc9628-8848-4d26-a815-ac11180adb5c None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:07.155877 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4b1ea1-571d-4dd1-96be-2e843ced28e1 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:07.158068 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 471/927] 10.222.0.82 () {56 vars in 1087 bytes} [Thu Aug 18 16:55:07 2022] GET /identity/v3/services => generated 1497 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 16:55:07.274383 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4b1ea1-571d-4dd1-96be-2e843ced28e1 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:07.301156 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:07.313855 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f287bb02-3978-455c-afad-d5c4c2cbc70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:07.314594 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f287bb02-3978-455c-afad-d5c4c2cbc70e None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:07.315003 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f287bb02-3978-455c-afad-d5c4c2cbc70e None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:07.315353 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f287bb02-3978-455c-afad-d5c4c2cbc70e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:07.316004 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f287bb02-3978-455c-afad-d5c4c2cbc70e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:07.317451 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f287bb02-3978-455c-afad-d5c4c2cbc70e None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:07.328802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-f287bb02-3978-455c-afad-d5c4c2cbc70e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 18 16:55:07.328802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:07.328802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 16:55:07.328802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 16:55:07.328802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 16:55:07.328802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 16:55:07.328802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 16:55:07.328802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 16:55:07.328802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 18 16:55:07.328802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:55:07.328802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 16:55:07.328802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:55:07.328802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:07.328802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:55:07.328802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:55:07.328802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:55:07.328802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:55:07.329808 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:55:07.329808 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:55:07.329808 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:55:07.329808 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:55:07.329808 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:55:07.329808 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:55:07.329808 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 16:55:07.329808 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 16:55:07.329808 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 16:55:07.329808 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 16:55:07.329808 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:55:07.329808 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:55:07.329808 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:55:07.329808 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:55:07.329808 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:55:07.329808 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:55:07.330629 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:55:07.330629 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:55:07.330629 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:55:07.330629 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:55:07.330629 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:55:07.330629 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:55:07.330629 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:55:07.330629 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:55:07.330629 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:55:07.330629 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:55:07.330629 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 16:55:07.330629 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 16:55:07.330629 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 18 16:55:07.330629 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:55:07.336347 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 457/928] 10.222.0.82 () {56 vars in 1114 bytes} [Thu Aug 18 16:55:07 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 16:55:07.337346 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbc9628-8848-4d26-a815-ac11180adb5c None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:07.441172 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:07.450535 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbc9628-8848-4d26-a815-ac11180adb5c None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:07.486376 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2c9d8c-58ef-4df4-bad1-027e14b4bbdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:07.487009 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c9d8c-58ef-4df4-bad1-027e14b4bbdf None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:07.487290 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c9d8c-58ef-4df4-bad1-027e14b4bbdf None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:07.487559 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c9d8c-58ef-4df4-bad1-027e14b4bbdf None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:07.488176 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2c9d8c-58ef-4df4-bad1-027e14b4bbdf None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:07.488474 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2c9d8c-58ef-4df4-bad1-027e14b4bbdf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:07.489827 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2c9d8c-58ef-4df4-bad1-027e14b4bbdf None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:07.500409 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 472/929] 10.222.0.82 () {56 vars in 1114 bytes} [Thu Aug 18 16:55:07 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:55:07.501756 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f287bb02-3978-455c-afad-d5c4c2cbc70e None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:07.606781 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f287bb02-3978-455c-afad-d5c4c2cbc70e None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:07.631369 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b043bdf-f13c-4ba7-a232-cf713bb9b8ac None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:07.631816 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b043bdf-f13c-4ba7-a232-cf713bb9b8ac None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:07.632218 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b043bdf-f13c-4ba7-a232-cf713bb9b8ac None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:07.633934 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 473/930] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:55:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:55:07.638852 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc0441-e34f-47e7-8cf5-52ed115bb91b None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:07.639350 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc0441-e34f-47e7-8cf5-52ed115bb91b None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:07.639721 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc0441-e34f-47e7-8cf5-52ed115bb91b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:07.644119 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0a481a-d6e8-4a87-b28d-ea1bb7aa88ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:07.644913 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a481a-d6e8-4a87-b28d-ea1bb7aa88ad None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:07.645345 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a481a-d6e8-4a87-b28d-ea1bb7aa88ad None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:07.645787 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a481a-d6e8-4a87-b28d-ea1bb7aa88ad None admin] PATH_INFO: `/v3/services` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:07.646621 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0a481a-d6e8-4a87-b28d-ea1bb7aa88ad None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:07.647097 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0a481a-d6e8-4a87-b28d-ea1bb7aa88ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:07.648614 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0a481a-d6e8-4a87-b28d-ea1bb7aa88ad None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:07.654287 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 458/931] 10.222.0.82 () {56 vars in 1114 bytes} [Thu Aug 18 16:55:07 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:07.661567 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-d7fc0441-e34f-47e7-8cf5-52ed115bb91b None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:55:07.661567 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-d7fc0441-e34f-47e7-8cf5-52ed115bb91b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:55:07.661795 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0a481a-d6e8-4a87-b28d-ea1bb7aa88ad None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:07.730692 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:07.747047 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 474/932] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:55:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 16:55:07.801496 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0a481a-d6e8-4a87-b28d-ea1bb7aa88ad None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:07.836538 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a568f-69d9-433d-af80-cb2e2da06ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:07.837253 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a568f-69d9-433d-af80-cb2e2da06ec2 None admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:07.837611 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a568f-69d9-433d-af80-cb2e2da06ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:07.838533 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a568f-69d9-433d-af80-cb2e2da06ec2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:07.839385 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1a568f-69d9-433d-af80-cb2e2da06ec2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:07.840840 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1a568f-69d9-433d-af80-cb2e2da06ec2 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:07.952855 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 459/933] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 16:55:07 2022] POST /identity/v3/endpoints => generated 346 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 16:55:08.036359 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc0441-e34f-47e7-8cf5-52ed115bb91b None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:08.128685 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:08.143924 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc0441-e34f-47e7-8cf5-52ed115bb91b None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:08.185666 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376808e4-2148-4f33-ba04-add741b944d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:08.186353 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376808e4-2148-4f33-ba04-add741b944d0 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:08.186638 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376808e4-2148-4f33-ba04-add741b944d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:08.186921 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376808e4-2148-4f33-ba04-add741b944d0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:08.197909 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376808e4-2148-4f33-ba04-add741b944d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:55:08.202775 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-376808e4-2148-4f33-ba04-add741b944d0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 18 16:55:08.202775 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:08.202775 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:55:08.202775 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:55:08.202775 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:55:08.202775 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:55:08.202775 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:55:08.202775 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:55:08.202775 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:55:08.202775 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:55:08.202775 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:55:08.202775 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:55:08.202775 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:55:08.202775 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:55:08.202775 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:55:08.202775 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:55:08.203682 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:55:08.203682 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:55:08.203682 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:55:08.203682 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:55:08.203682 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:55:08.203682 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:55:08.203682 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:55:08.203682 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:55:08.203682 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:55:08.203682 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:55:08.203682 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:55:08.203682 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:55:08.203682 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:55:08.203682 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:55:08.203682 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:55:08.203682 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:55:08.204881 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:55:08.204881 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:55:08.204881 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:55:08.204881 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:55:08.204881 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:55:08.204881 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:55:08.204881 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:55:08.204881 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:55:08.204881 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:55:08.204881 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:55:08.204881 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 18 16:55:08.204881 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:55:08.204881 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 475/934] 10.222.0.82 () {56 vars in 1120 bytes} [Thu Aug 18 16:55:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:55:08.210154 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a568f-69d9-433d-af80-cb2e2da06ec2 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:08.330817 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a568f-69d9-433d-af80-cb2e2da06ec2 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:08.340811 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:08.372506 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f43c5e3-3848-4861-9bff-019742e93b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:08.373142 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f43c5e3-3848-4861-9bff-019742e93b49 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:08.373598 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f43c5e3-3848-4861-9bff-019742e93b49 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:08.373875 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f43c5e3-3848-4861-9bff-019742e93b49 None admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:08.374547 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f43c5e3-3848-4861-9bff-019742e93b49 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:08.374853 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f43c5e3-3848-4861-9bff-019742e93b49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:08.376325 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f43c5e3-3848-4861-9bff-019742e93b49 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:08.386926 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 460/935] 10.222.0.82 () {56 vars in 1115 bytes} [Thu Aug 18 16:55:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:08.392302 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376808e4-2148-4f33-ba04-add741b944d0 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:08.501920 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376808e4-2148-4f33-ba04-add741b944d0 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:08.537666 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0086a82-6822-4d6e-9309-a24d76c86a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:08.538665 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0086a82-6822-4d6e-9309-a24d76c86a77 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:08.539238 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0086a82-6822-4d6e-9309-a24d76c86a77 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:08.539726 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0086a82-6822-4d6e-9309-a24d76c86a77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:08.551351 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0086a82-6822-4d6e-9309-a24d76c86a77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:08.553017 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0086a82-6822-4d6e-9309-a24d76c86a77 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:08.555090 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:08.571749 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-b0086a82-6822-4d6e-9309-a24d76c86a77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:55:08.571749 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:08.571749 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:55:08.571749 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:55:08.571749 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:55:08.571749 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:55:08.571749 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:55:08.571749 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:55:08.571749 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:55:08.571749 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:55:08.571749 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:55:08.571749 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:55:08.571749 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:55:08.571749 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:55:08.571749 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:55:08.571749 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:55:08.578257 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:55:08.578257 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:55:08.578257 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:55:08.578257 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:55:08.578257 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:55:08.578257 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:55:08.578257 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:55:08.578257 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:55:08.578257 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:55:08.578257 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:55:08.578257 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:55:08.578257 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:55:08.578257 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:55:08.578257 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:55:08.578257 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:55:08.578257 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:55:08.578257 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:55:08.579258 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:55:08.579258 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:55:08.579258 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:55:08.579258 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 476/936] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:55:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:55:08.581702 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f43c5e3-3848-4861-9bff-019742e93b49 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:08.687205 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f43c5e3-3848-4861-9bff-019742e93b49 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:08.722623 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc8a3ca-517a-4de9-a29c-1f40d4725fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:08.723274 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc8a3ca-517a-4de9-a29c-1f40d4725fbd None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:08.723558 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc8a3ca-517a-4de9-a29c-1f40d4725fbd None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:08.723833 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc8a3ca-517a-4de9-a29c-1f40d4725fbd None admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:08.724476 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc8a3ca-517a-4de9-a29c-1f40d4725fbd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:08.724786 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc8a3ca-517a-4de9-a29c-1f40d4725fbd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:08.726175 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc8a3ca-517a-4de9-a29c-1f40d4725fbd None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:08.738714 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 461/937] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:55:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:08.744981 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0086a82-6822-4d6e-9309-a24d76c86a77 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:08.825770 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a721e82-8d65-4ede-8bac-b2524f380557 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:08.826134 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a721e82-8d65-4ede-8bac-b2524f380557 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:08.826337 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a721e82-8d65-4ede-8bac-b2524f380557 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:08.827952 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 462/938] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:55:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 16:55:08.833204 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc3422-b846-4dec-9119-2351667e9625 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:08.833730 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc3422-b846-4dec-9119-2351667e9625 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:08.834248 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc3422-b846-4dec-9119-2351667e9625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:08.853984 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-d8dc3422-b846-4dec-9119-2351667e9625 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:55:08.855139 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0086a82-6822-4d6e-9309-a24d76c86a77 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:08.855880 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-d8dc3422-b846-4dec-9119-2351667e9625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:55:08.888173 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:08.892746 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f58d79a-0418-4172-bb99-31a159527251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:08.893391 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f58d79a-0418-4172-bb99-31a159527251 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:08.893671 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f58d79a-0418-4172-bb99-31a159527251 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:08.893942 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f58d79a-0418-4172-bb99-31a159527251 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:08.904350 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f58d79a-0418-4172-bb99-31a159527251 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:08.908174 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f58d79a-0418-4172-bb99-31a159527251 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:08.920358 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-7f58d79a-0418-4172-bb99-31a159527251 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 18 16:55:08.920358 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:08.920358 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:55:08.920358 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:55:08.920358 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:55:08.920358 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:55:08.920358 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:55:08.920358 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:55:08.920358 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:55:08.920358 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:55:08.920358 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:55:08.920358 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:55:08.920358 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 16:55:08.920358 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 16:55:08.920358 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 16:55:08.920358 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 16:55:08.921554 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:55:08.921554 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:55:08.921554 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 16:55:08.921554 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:55:08.921554 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 16:55:08.921554 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:55:08.921554 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:55:08.921554 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:55:08.921554 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:55:08.921554 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:55:08.921554 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:55:08.921554 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:55:08.921554 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:55:08.921554 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:55:08.921554 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:55:08.921554 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:55:08.921554 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:55:08.921554 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:55:08.922868 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:55:08.922868 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:55:08.922868 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 16:55:08.922868 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 16:55:08.922868 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 16:55:08.922868 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 16:55:08.922868 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 16:55:08.922868 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 16:55:08.922868 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 18 16:55:08.922868 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:55:08.926806 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 477/939] 10.222.0.82 () {56 vars in 1117 bytes} [Thu Aug 18 16:55:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:55:08.928536 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f58d79a-0418-4172-bb99-31a159527251 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:08.951001 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 463/940] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:55:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 16:55:08.996949 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:09.042290 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f58d79a-0418-4172-bb99-31a159527251 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:09.077570 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810b9956-895e-4d37-8cdf-769d48a15a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:09.078250 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810b9956-895e-4d37-8cdf-769d48a15a3b None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:09.078532 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810b9956-895e-4d37-8cdf-769d48a15a3b None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:09.078807 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810b9956-895e-4d37-8cdf-769d48a15a3b None admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:09.079565 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810b9956-895e-4d37-8cdf-769d48a15a3b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:09.079868 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810b9956-895e-4d37-8cdf-769d48a15a3b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:09.081293 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810b9956-895e-4d37-8cdf-769d48a15a3b None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:09.099665 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 478/941] 10.222.0.82 () {56 vars in 1149 bytes} [Thu Aug 18 16:55:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:09.105666 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dc3422-b846-4dec-9119-2351667e9625 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:09.213945 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dc3422-b846-4dec-9119-2351667e9625 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:09.235384 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810b9956-895e-4d37-8cdf-769d48a15a3b None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:09.251343 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc98f8b-bb91-474d-816e-0fdade33a193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:09.251987 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc98f8b-bb91-474d-816e-0fdade33a193 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:09.252273 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc98f8b-bb91-474d-816e-0fdade33a193 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:09.252549 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc98f8b-bb91-474d-816e-0fdade33a193 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:09.263783 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc98f8b-bb91-474d-816e-0fdade33a193 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:09.265236 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc98f8b-bb91-474d-816e-0fdade33a193 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:09.280385 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-8fc98f8b-bb91-474d-816e-0fdade33a193 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:55:09.280385 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:09.280385 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:55:09.280385 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:55:09.280385 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:55:09.280385 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:55:09.280385 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:55:09.280385 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:55:09.280385 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:55:09.280385 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:55:09.280385 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:55:09.280385 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:55:09.280385 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:55:09.280385 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:55:09.280385 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:55:09.280385 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:55:09.281326 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:55:09.281326 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:55:09.281326 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:55:09.281326 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:55:09.281326 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:55:09.281326 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:55:09.281326 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:55:09.281326 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:55:09.281326 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:55:09.281326 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:55:09.281326 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:55:09.281326 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:55:09.281326 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:55:09.281326 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:55:09.281326 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:55:09.281326 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:55:09.281326 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:55:09.282212 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:55:09.282212 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:55:09.282212 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:55:09.282212 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 464/942] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:55:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:55:09.283031 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2006564c-75d8-4c0e-8b4f-dec4b0f1c9b8 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:09.283320 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2006564c-75d8-4c0e-8b4f-dec4b0f1c9b8 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:09.283598 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2006564c-75d8-4c0e-8b4f-dec4b0f1c9b8 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:09.284960 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 465/943] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:55:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:55:09.287729 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2006564c-75d8-4c0e-8b4f-dec4b0f1c9b8 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:09.302263 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:09.355388 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810b9956-895e-4d37-8cdf-769d48a15a3b None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:09.369223 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:09.390600 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae1cc9f-90dc-4df7-8891-f4274b2fba7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:09.392258 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae1cc9f-90dc-4df7-8891-f4274b2fba7a None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:09.392258 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae1cc9f-90dc-4df7-8891-f4274b2fba7a None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:09.392258 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae1cc9f-90dc-4df7-8891-f4274b2fba7a None admin] PATH_INFO: `/v3/services/compute` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:09.392947 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae1cc9f-90dc-4df7-8891-f4274b2fba7a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:09.393791 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae1cc9f-90dc-4df7-8891-f4274b2fba7a None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:09.398011 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2006564c-75d8-4c0e-8b4f-dec4b0f1c9b8 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:09.403424 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-6ae1cc9f-90dc-4df7-8891-f4274b2fba7a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 18 16:55:09.403424 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:09.403424 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 16:55:09.403424 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 16:55:09.403424 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 16:55:09.403424 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 16:55:09.403424 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 16:55:09.403424 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 16:55:09.403424 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 18 16:55:09.403424 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:55:09.403424 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 16:55:09.403424 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:55:09.403424 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:09.403424 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:55:09.403424 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:55:09.403424 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:55:09.403424 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:55:09.404465 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:55:09.404465 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:55:09.404465 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:55:09.404465 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:55:09.404465 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:55:09.404465 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:55:09.404465 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 16:55:09.404465 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 16:55:09.404465 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 16:55:09.404465 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 16:55:09.404465 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:55:09.404465 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:55:09.404465 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:55:09.404465 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:55:09.404465 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:55:09.404465 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:55:09.405281 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:55:09.405281 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:55:09.405281 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:55:09.405281 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:55:09.405281 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:55:09.405281 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:55:09.405281 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:55:09.405281 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:55:09.405281 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:55:09.405281 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:55:09.405281 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 16:55:09.405281 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 16:55:09.405281 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 18 16:55:09.405281 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:55:09.405281 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 479/944] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 16:55:09 2022] GET /identity/v3/services/compute => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 16:55:09.406105 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9ee7f4-0034-458e-83af-34f207cccab9 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:09.406735 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9ee7f4-0034-458e-83af-34f207cccab9 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:09.406735 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9ee7f4-0034-458e-83af-34f207cccab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:09.426733 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-1b9ee7f4-0034-458e-83af-34f207cccab9 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:55:09.428065 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-1b9ee7f4-0034-458e-83af-34f207cccab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:55:09.439654 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168fc82b-66be-42a0-8e34-d590d147b1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:09.440317 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168fc82b-66be-42a0-8e34-d590d147b1a9 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:09.440605 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168fc82b-66be-42a0-8e34-d590d147b1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:09.440930 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168fc82b-66be-42a0-8e34-d590d147b1a9 None admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:09.441690 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168fc82b-66be-42a0-8e34-d590d147b1a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:09.441998 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168fc82b-66be-42a0-8e34-d590d147b1a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:09.443425 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168fc82b-66be-42a0-8e34-d590d147b1a9 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:09.455327 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 466/945] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:55:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:09.455327 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168fc82b-66be-42a0-8e34-d590d147b1a9 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:09.506405 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 480/946] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:55:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 16:55:09.507550 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9ee7f4-0034-458e-83af-34f207cccab9 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:09.542643 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:09.574920 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168fc82b-66be-42a0-8e34-d590d147b1a9 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:09.617814 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9ee7f4-0034-458e-83af-34f207cccab9 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:09.621859 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f8f34f-1242-453a-8123-d42296a79688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:09.622794 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8f34f-1242-453a-8123-d42296a79688 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:09.623338 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8f34f-1242-453a-8123-d42296a79688 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:09.623858 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8f34f-1242-453a-8123-d42296a79688 None admin] PATH_INFO: `/v3/services` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:09.624687 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f8f34f-1242-453a-8123-d42296a79688 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:09.625224 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f8f34f-1242-453a-8123-d42296a79688 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:09.625920 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:09.627070 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f8f34f-1242-453a-8123-d42296a79688 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:09.639193 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 467/947] 10.222.0.82 () {56 vars in 1112 bytes} [Thu Aug 18 16:55:09 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:09.640387 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f8f34f-1242-453a-8123-d42296a79688 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:09.658402 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b3d10b-8d90-4085-a1a2-f247ad01f353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:09.659213 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b3d10b-8d90-4085-a1a2-f247ad01f353 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:09.660525 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b3d10b-8d90-4085-a1a2-f247ad01f353 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:09.661078 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b3d10b-8d90-4085-a1a2-f247ad01f353 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:09.672811 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b3d10b-8d90-4085-a1a2-f247ad01f353 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:09.674967 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b3d10b-8d90-4085-a1a2-f247ad01f353 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:09.694281 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-e2b3d10b-8d90-4085-a1a2-f247ad01f353 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:55:09.694281 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:09.694281 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:55:09.694281 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:55:09.694281 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:55:09.694281 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:55:09.694281 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:55:09.694281 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:55:09.694281 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:55:09.694281 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:55:09.694281 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:55:09.694281 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:55:09.694281 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 16:55:09.694281 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 16:55:09.694281 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 16:55:09.695484 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 16:55:09.695484 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:55:09.695484 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:55:09.695484 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:55:09.695484 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:55:09.695484 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:55:09.695484 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:55:09.695484 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:55:09.695484 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:55:09.695484 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:55:09.695484 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:55:09.695484 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:55:09.695484 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:55:09.695484 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:55:09.695484 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:55:09.695484 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:55:09.695484 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:55:09.696422 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 16:55:09.696422 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 16:55:09.696422 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 16:55:09.696422 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 16:55:09.696422 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 16:55:09.696422 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 16:55:09.696422 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:55:09.696422 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:55:09.698970 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 481/948] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 16:55:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:55:09.701841 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b3d10b-8d90-4085-a1a2-f247ad01f353 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:09.769048 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f8f34f-1242-453a-8123-d42296a79688 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:09.807584 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f7bba8-6057-40ed-9a25-40079489536f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:09.808590 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f7bba8-6057-40ed-9a25-40079489536f None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:09.809109 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f7bba8-6057-40ed-9a25-40079489536f None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:09.809385 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f7bba8-6057-40ed-9a25-40079489536f None admin] PATH_INFO: `/v3/services` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:09.810011 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f7bba8-6057-40ed-9a25-40079489536f None admin] RBAC: Adding query filter params (type=compute) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:09.810320 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f7bba8-6057-40ed-9a25-40079489536f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:09.811839 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f7bba8-6057-40ed-9a25-40079489536f None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:09.813319 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:09.817635 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 468/949] 10.222.0.82 () {56 vars in 1112 bytes} [Thu Aug 18 16:55:09 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:09.818635 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f7bba8-6057-40ed-9a25-40079489536f None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:09.827809 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b3d10b-8d90-4085-a1a2-f247ad01f353 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:09.881685 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdaa6fc-a253-437e-863e-e470948efe64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:09.883485 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdaa6fc-a253-437e-863e-e470948efe64 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:09.883970 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdaa6fc-a253-437e-863e-e470948efe64 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:09.885270 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdaa6fc-a253-437e-863e-e470948efe64 None admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:09.886189 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdaa6fc-a253-437e-863e-e470948efe64 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:09.886992 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdaa6fc-a253-437e-863e-e470948efe64 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:09.889498 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdaa6fc-a253-437e-863e-e470948efe64 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:09.913713 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 482/950] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 16:55:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:55:09.914297 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdaa6fc-a253-437e-863e-e470948efe64 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:09.945568 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f7bba8-6057-40ed-9a25-40079489536f None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:09.948144 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:09.985751 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dc5e12-bfc8-46a3-83d1-c245e1694d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:09.986770 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc5e12-bfc8-46a3-83d1-c245e1694d64 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:09.987314 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc5e12-bfc8-46a3-83d1-c245e1694d64 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:09.987801 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc5e12-bfc8-46a3-83d1-c245e1694d64 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:09.988593 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dc5e12-bfc8-46a3-83d1-c245e1694d64 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:09.990192 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dc5e12-bfc8-46a3-83d1-c245e1694d64 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:09.999896 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-87dc5e12-bfc8-46a3-83d1-c245e1694d64 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 16:55:09.999896 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:09.999896 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 16:55:09.999896 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 16:55:09.999896 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 16:55:09.999896 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 16:55:09.999896 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 16:55:09.999896 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 16:55:09.999896 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 16:55:09.999896 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:55:09.999896 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 16:55:09.999896 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:55:09.999896 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:09.999896 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:55:09.999896 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:55:09.999896 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:55:09.999896 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:55:10.000908 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:55:10.000908 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:55:10.000908 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:55:10.000908 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:55:10.000908 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:55:10.000908 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:55:10.000908 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 16:55:10.000908 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 16:55:10.000908 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 16:55:10.000908 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 16:55:10.000908 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:55:10.000908 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:55:10.000908 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:55:10.000908 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:55:10.000908 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:55:10.000908 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:55:10.001707 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:55:10.001707 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:55:10.001707 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:55:10.001707 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:55:10.001707 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:55:10.001707 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:55:10.001707 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:55:10.001707 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:55:10.001707 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:55:10.001707 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:55:10.001707 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 16:55:10.001707 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 16:55:10.001707 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 16:55:10.001707 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:55:10.001707 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 469/951] 10.222.0.82 () {56 vars in 1126 bytes} [Thu Aug 18 16:55:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:55:10.007013 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17903483-f357-4900-9166-2cfefb521aea None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:10.007013 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17903483-f357-4900-9166-2cfefb521aea None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:10.007013 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17903483-f357-4900-9166-2cfefb521aea None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:10.007013 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 470/952] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:55:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 16:55:10.008477 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17903483-f357-4900-9166-2cfefb521aea None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:10.039910 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdaa6fc-a253-437e-863e-e470948efe64 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:10.074430 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:10.079982 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d649b556-4bf7-4fce-8f71-ce2b0c72c324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:10.080655 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d649b556-4bf7-4fce-8f71-ce2b0c72c324 None admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:10.080939 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d649b556-4bf7-4fce-8f71-ce2b0c72c324 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:10.081245 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d649b556-4bf7-4fce-8f71-ce2b0c72c324 None admin] PATH_INFO: `/v3/projects/2293c0aab61f44849d0d48b214cde7ea/users/5de19a54ed104bcdb35ee4e5e793f1e7/roles/bae4160aa9de4d69b42641fd18c53e55` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:10.083400 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d649b556-4bf7-4fce-8f71-ce2b0c72c324 None admin] RBAC: Authorizing `identity:create_grant(project_id=2293c0aab61f44849d0d48b214cde7ea, user_id=5de19a54ed104bcdb35ee4e5e793f1e7, role_id=bae4160aa9de4d69b42641fd18c53e55)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:10.085824 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d649b556-4bf7-4fce-8f71-ce2b0c72c324 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:10.114308 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17903483-f357-4900-9166-2cfefb521aea None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:10.148426 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dd3f91-18e4-4908-a41c-6eb2ea6ddc6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:10.148959 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd3f91-18e4-4908-a41c-6eb2ea6ddc6e None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:10.149238 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd3f91-18e4-4908-a41c-6eb2ea6ddc6e None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:10.149513 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd3f91-18e4-4908-a41c-6eb2ea6ddc6e None admin] PATH_INFO: `/v3/services` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:10.150230 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dd3f91-18e4-4908-a41c-6eb2ea6ddc6e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:10.150529 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dd3f91-18e4-4908-a41c-6eb2ea6ddc6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:10.151954 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dd3f91-18e4-4908-a41c-6eb2ea6ddc6e None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:10.156673 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 471/953] 10.222.0.82 () {56 vars in 1122 bytes} [Thu Aug 18 16:55:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:55:10.157919 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a9925-db3c-45d2-bf76-dd057a90ef44 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:10.158096 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a9925-db3c-45d2-bf76-dd057a90ef44 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:10.158371 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a9925-db3c-45d2-bf76-dd057a90ef44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:10.175731 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-693a9925-db3c-45d2-bf76-dd057a90ef44 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:55:10.176913 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-693a9925-db3c-45d2-bf76-dd057a90ef44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:55:10.191265 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 483/954] 10.222.0.82 () {58 vars in 1439 bytes} [Thu Aug 18 16:55:09 2022] PUT /identity/v3/projects/2293c0aab61f44849d0d48b214cde7ea/users/5de19a54ed104bcdb35ee4e5e793f1e7/roles/bae4160aa9de4d69b42641fd18c53e55 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 16:55:10.192179 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d649b556-4bf7-4fce-8f71-ce2b0c72c324 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:10.265392 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 472/955] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:55:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 16:55:10.298147 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:10.316483 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d649b556-4bf7-4fce-8f71-ce2b0c72c324 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:10.357531 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9ec803-3422-4132-b83f-e7ea60d6b396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:10.358155 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9ec803-3422-4132-b83f-e7ea60d6b396 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:10.358437 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9ec803-3422-4132-b83f-e7ea60d6b396 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:10.358707 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9ec803-3422-4132-b83f-e7ea60d6b396 None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:10.359424 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9ec803-3422-4132-b83f-e7ea60d6b396 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:10.359729 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9ec803-3422-4132-b83f-e7ea60d6b396 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:10.361102 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9ec803-3422-4132-b83f-e7ea60d6b396 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:10.366255 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 484/956] 10.222.0.82 () {56 vars in 1122 bytes} [Thu Aug 18 16:55:10 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:10.372324 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693a9925-db3c-45d2-bf76-dd057a90ef44 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:10.436048 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:10.481834 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693a9925-db3c-45d2-bf76-dd057a90ef44 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:10.517551 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1989193e-287f-4ab7-ab6d-71ab6389079b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:10.518231 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989193e-287f-4ab7-ab6d-71ab6389079b None admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:10.518519 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989193e-287f-4ab7-ab6d-71ab6389079b None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:10.518794 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989193e-287f-4ab7-ab6d-71ab6389079b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:10.519486 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1989193e-287f-4ab7-ab6d-71ab6389079b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:10.520914 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1989193e-287f-4ab7-ab6d-71ab6389079b None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:10.547085 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9ec803-3422-4132-b83f-e7ea60d6b396 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:10.599586 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 473/957] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 16:55:10 2022] POST /identity/v3/endpoints => generated 349 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 16:55:10.678453 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9ec803-3422-4132-b83f-e7ea60d6b396 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:10.727065 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5501062-c868-462c-944c-e3ed226276d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:10.727602 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5501062-c868-462c-944c-e3ed226276d5 None admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:10.727886 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5501062-c868-462c-944c-e3ed226276d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:10.728206 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5501062-c868-462c-944c-e3ed226276d5 None admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:10.729158 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5501062-c868-462c-944c-e3ed226276d5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:10.730651 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5501062-c868-462c-944c-e3ed226276d5 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:10.733444 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: WARNING keystone.server.flask.common [None req-f5501062-c868-462c-944c-e3ed226276d5 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 18 16:55:10.776969 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.sql.core [None req-f5501062-c868-462c-944c-e3ed226276d5 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 18 16:55:10.776969 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 18 16:55:10.776969 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [parameters: {'id': '64ba5da317314fba86585dea27e898a4', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 18 16:55:10.776969 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81649) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 18 16:55:10.793781 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-f5501062-c868-462c-944c-e3ed226276d5 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 64ba5da317314fba86585dea27e898a4.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 64ba5da317314fba86585dea27e898a4. Aug 18 16:55:10.793781 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:10.793781 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 18 16:55:10.793781 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 18 16:55:10.793781 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 18 16:55:10.793781 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 18 16:55:10.793781 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 18 16:55:10.793781 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application result = self._query(query) Aug 18 16:55:10.793781 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 18 16:55:10.793781 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application conn.query(q) Aug 18 16:55:10.793781 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 18 16:55:10.793781 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 18 16:55:10.793781 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 18 16:55:10.794787 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application result.read() Aug 18 16:55:10.794787 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 18 16:55:10.794787 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 18 16:55:10.794787 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 18 16:55:10.794787 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 18 16:55:10.794787 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 18 16:55:10.794787 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 18 16:55:10.794787 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 18 16:55:10.794787 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 18 16:55:10.794787 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 18 16:55:10.794787 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:55:10.794787 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 18 16:55:10.794787 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:55:10.794787 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:10.794787 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 18 16:55:10.794787 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 18 16:55:10.794787 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 18 16:55:10.794787 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 18 16:55:10.795781 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 18 16:55:10.795781 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application next(self.gen) Aug 18 16:55:10.795781 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 18 16:55:10.795781 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application yield resource Aug 18 16:55:10.795781 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 18 16:55:10.795781 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application next(self.gen) Aug 18 16:55:10.795781 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 18 16:55:10.795781 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self.session.rollback() Aug 18 16:55:10.795781 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 18 16:55:10.795781 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self.force_reraise() Aug 18 16:55:10.795781 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 18 16:55:10.795781 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise self.value Aug 18 16:55:10.795781 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 18 16:55:10.795781 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 18 16:55:10.795781 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 18 16:55:10.795781 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application session.commit() Aug 18 16:55:10.795781 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 18 16:55:10.796639 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 18 16:55:10.796639 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 18 16:55:10.796639 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self._prepare_impl() Aug 18 16:55:10.796639 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 18 16:55:10.796639 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self.session.flush() Aug 18 16:55:10.796639 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 18 16:55:10.796639 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self._flush(objects) Aug 18 16:55:10.796639 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 18 16:55:10.796639 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 18 16:55:10.796639 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 18 16:55:10.796639 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application compat.raise_( Aug 18 16:55:10.796639 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 18 16:55:10.796639 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception Aug 18 16:55:10.796639 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 18 16:55:10.796639 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application flush_context.execute() Aug 18 16:55:10.796639 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 18 16:55:10.796639 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rec.execute(self) Aug 18 16:55:10.797561 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 18 16:55:10.797561 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 18 16:55:10.797561 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 18 16:55:10.797561 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 18 16:55:10.797561 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 18 16:55:10.797561 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 18 16:55:10.797561 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 18 16:55:10.797561 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 18 16:55:10.797561 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 18 16:55:10.797561 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 18 16:55:10.797561 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 18 16:55:10.797561 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 18 16:55:10.797561 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 18 16:55:10.797561 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 18 16:55:10.797561 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 18 16:55:10.798389 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 18 16:55:10.798389 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 18 16:55:10.798389 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception Aug 18 16:55:10.798389 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 18 16:55:10.798389 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 18 16:55:10.798389 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 18 16:55:10.798389 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 18 16:55:10.798389 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 18 16:55:10.798389 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application result = self._query(query) Aug 18 16:55:10.798389 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 18 16:55:10.798389 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application conn.query(q) Aug 18 16:55:10.798389 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 18 16:55:10.798389 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 18 16:55:10.798389 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 18 16:55:10.798389 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application result.read() Aug 18 16:55:10.798389 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 18 16:55:10.798389 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 18 16:55:10.800499 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 18 16:55:10.800499 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 18 16:55:10.800499 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 18 16:55:10.800499 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 18 16:55:10.800499 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 18 16:55:10.800499 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 18 16:55:10.800499 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 18 16:55:10.800499 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 18 16:55:10.800499 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [parameters: {'id': '64ba5da317314fba86585dea27e898a4', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 18 16:55:10.800499 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 18 16:55:10.800499 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:55:10.800499 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 16:55:10.800499 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:55:10.800499 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:10.800499 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 18 16:55:10.801314 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 18 16:55:10.801314 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 18 16:55:10.801314 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 18 16:55:10.801314 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 18 16:55:10.801314 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:55:10.801314 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 16:55:10.801314 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:55:10.801314 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:10.801314 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:55:10.801314 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:55:10.801314 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:55:10.801314 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:55:10.801314 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:55:10.801314 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:55:10.801314 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:55:10.801314 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:55:10.801314 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:55:10.802251 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:55:10.802251 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 18 16:55:10.802251 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 18 16:55:10.802251 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:55:10.802251 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:55:10.802251 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 18 16:55:10.802251 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 18 16:55:10.802251 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 64ba5da317314fba86585dea27e898a4. Aug 18 16:55:10.802251 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:55:10.802251 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 485/958] 10.222.0.82 () {60 vars in 1140 bytes} [Thu Aug 18 16:55:10 2022] POST /identity/v3/projects => generated 262 bytes in 249 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 18 16:55:10.802792 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1989193e-287f-4ab7-ab6d-71ab6389079b None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:10.917922 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1989193e-287f-4ab7-ab6d-71ab6389079b None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:10.932305 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:10.954416 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597b5037-c60f-451c-b18c-10111471ac2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:10.955096 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b5037-c60f-451c-b18c-10111471ac2c None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:10.955365 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b5037-c60f-451c-b18c-10111471ac2c None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:10.955636 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b5037-c60f-451c-b18c-10111471ac2c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:10.963033 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597b5037-c60f-451c-b18c-10111471ac2c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:10.964868 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597b5037-c60f-451c-b18c-10111471ac2c None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:10.977866 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-597b5037-c60f-451c-b18c-10111471ac2c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:55:10.977866 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:10.977866 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:55:10.977866 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:55:10.977866 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:55:10.977866 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:55:10.977866 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:55:10.977866 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:55:10.977866 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:55:10.977866 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:55:10.977866 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:55:10.977866 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:55:10.977866 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 16:55:10.977866 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 16:55:10.977866 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 16:55:10.980399 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 16:55:10.980399 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:55:10.980399 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:55:10.980399 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:55:10.980399 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:55:10.980399 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:55:10.980399 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:55:10.980399 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:55:10.980399 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:55:10.980399 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:55:10.980399 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:55:10.980399 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:55:10.980399 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:55:10.980399 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:55:10.980399 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:55:10.980399 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:55:10.980399 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:55:10.982507 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 16:55:10.982507 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 16:55:10.982507 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 16:55:10.982507 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 16:55:10.982507 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 16:55:10.982507 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 16:55:10.982507 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:55:10.982507 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:55:10.982507 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 474/959] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 16:55:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:55:10.989989 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5501062-c868-462c-944c-e3ed226276d5 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:11.007344 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:11.096574 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5501062-c868-462c-944c-e3ed226276d5 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:11.134108 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3636c79d-37d1-46b7-b291-570363d55595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:11.135896 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636c79d-37d1-46b7-b291-570363d55595 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:11.136407 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636c79d-37d1-46b7-b291-570363d55595 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:11.136875 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636c79d-37d1-46b7-b291-570363d55595 None admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:11.137725 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3636c79d-37d1-46b7-b291-570363d55595 None admin] RBAC: Adding query filter params (name=service) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:11.138230 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3636c79d-37d1-46b7-b291-570363d55595 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:11.140150 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3636c79d-37d1-46b7-b291-570363d55595 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:11.166947 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 486/960] 10.222.0.82 () {56 vars in 1112 bytes} [Thu Aug 18 16:55:10 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:11.825699 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224c3ac-faba-4aba-a2b4-44294f1ff40f None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:11.826203 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224c3ac-faba-4aba-a2b4-44294f1ff40f None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:11.826276 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224c3ac-faba-4aba-a2b4-44294f1ff40f None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:11.827854 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 475/961] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:55:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:55:11.833336 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c2b5a8-9b88-4121-98c7-aeade1707abf None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:11.833738 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c2b5a8-9b88-4121-98c7-aeade1707abf None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:11.833916 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c2b5a8-9b88-4121-98c7-aeade1707abf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:11.857757 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-54c2b5a8-9b88-4121-98c7-aeade1707abf None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:55:11.859293 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-54c2b5a8-9b88-4121-98c7-aeade1707abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:55:11.918213 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:11.928652 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 487/962] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:55:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 16:55:12.198875 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b224c3ac-faba-4aba-a2b4-44294f1ff40f None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:12.218605 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d682c17-ff80-4c01-95b3-3cb171334978 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:12.218938 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d682c17-ff80-4c01-95b3-3cb171334978 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:12.219200 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d682c17-ff80-4c01-95b3-3cb171334978 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:12.220811 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 488/963] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:55:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:55:12.228228 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f89f4-8f0a-41b5-9025-308a934baf83 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:12.228965 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f89f4-8f0a-41b5-9025-308a934baf83 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:12.229085 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f89f4-8f0a-41b5-9025-308a934baf83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:12.251709 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-bf7f89f4-8f0a-41b5-9025-308a934baf83 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:55:12.253793 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-bf7f89f4-8f0a-41b5-9025-308a934baf83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:55:12.323787 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b224c3ac-faba-4aba-a2b4-44294f1ff40f None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:12.334118 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 489/964] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:55:12 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 16:55:12.365620 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b99decf-a0ae-4c3a-b3c3-fb2a8d392bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:12.366540 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99decf-a0ae-4c3a-b3c3-fb2a8d392bc8 None admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:12.367020 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99decf-a0ae-4c3a-b3c3-fb2a8d392bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:12.367453 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99decf-a0ae-4c3a-b3c3-fb2a8d392bc8 None admin] PATH_INFO: `/v3/services` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:12.368290 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b99decf-a0ae-4c3a-b3c3-fb2a8d392bc8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:12.369957 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b99decf-a0ae-4c3a-b3c3-fb2a8d392bc8 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:12.440895 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 476/965] 10.222.0.82 () {60 vars in 1141 bytes} [Thu Aug 18 16:55:12 2022] POST /identity/v3/services => generated 242 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 16:55:12.628309 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7f89f4-8f0a-41b5-9025-308a934baf83 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:12.648416 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c4946-ab53-4368-9fa4-dc05fd78a72e None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:12.648750 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c4946-ab53-4368-9fa4-dc05fd78a72e None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:12.648975 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c4946-ab53-4368-9fa4-dc05fd78a72e None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:12.651528 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 477/966] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:55:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 16:55:12.656566 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea487c0-2489-4930-9e28-d1062f880b1a None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:12.656850 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea487c0-2489-4930-9e28-d1062f880b1a None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:12.657126 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea487c0-2489-4930-9e28-d1062f880b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:12.676053 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-2ea487c0-2489-4930-9e28-d1062f880b1a None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:55:12.677572 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-2ea487c0-2489-4930-9e28-d1062f880b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:55:12.705565 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:12.746319 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7f89f4-8f0a-41b5-9025-308a934baf83 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:12.759170 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 478/967] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:55:12 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 16:55:12.780060 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:12.798544 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a169b53-d3a7-499b-9f48-dc65015cb425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:12.799292 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a169b53-d3a7-499b-9f48-dc65015cb425 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:12.799570 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a169b53-d3a7-499b-9f48-dc65015cb425 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:12.799839 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a169b53-d3a7-499b-9f48-dc65015cb425 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:12.811385 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a169b53-d3a7-499b-9f48-dc65015cb425 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:55:12.818516 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-2a169b53-d3a7-499b-9f48-dc65015cb425 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 18 16:55:12.818516 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:12.818516 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:55:12.818516 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:55:12.818516 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:55:12.818516 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:55:12.818516 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:55:12.818516 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:55:12.818516 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:55:12.818516 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:55:12.818516 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:55:12.818516 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:55:12.818516 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:55:12.818516 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:55:12.818516 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:55:12.818516 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:55:12.819502 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:55:12.819502 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:55:12.819502 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:55:12.819502 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:55:12.819502 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:55:12.819502 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:55:12.819502 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:55:12.819502 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:55:12.819502 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:55:12.819502 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:55:12.819502 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:55:12.819502 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:55:12.819502 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:55:12.819502 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:55:12.819502 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:55:12.819502 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:55:12.820351 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:55:12.820351 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:55:12.820351 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:55:12.820351 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:55:12.820351 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:55:12.820351 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:55:12.820351 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:55:12.820351 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:55:12.820351 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:55:12.820351 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:55:12.820351 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 18 16:55:12.820351 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:55:12.820351 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 490/968] 10.222.0.82 () {56 vars in 1120 bytes} [Thu Aug 18 16:55:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:55:12.826734 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea487c0-2489-4930-9e28-d1062f880b1a None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:12.933499 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea487c0-2489-4930-9e28-d1062f880b1a None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:12.969370 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e46f2ce-e9d7-419d-9a36-7be8fb280294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:12.970363 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e46f2ce-e9d7-419d-9a36-7be8fb280294 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:12.970863 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e46f2ce-e9d7-419d-9a36-7be8fb280294 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:12.971387 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e46f2ce-e9d7-419d-9a36-7be8fb280294 None admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:12.972242 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e46f2ce-e9d7-419d-9a36-7be8fb280294 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:12.972744 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e46f2ce-e9d7-419d-9a36-7be8fb280294 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:12.974370 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e46f2ce-e9d7-419d-9a36-7be8fb280294 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:12.983240 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 479/969] 10.222.0.82 () {56 vars in 1115 bytes} [Thu Aug 18 16:55:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:12.991182 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a169b53-d3a7-499b-9f48-dc65015cb425 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:13.063649 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e46f2ce-e9d7-419d-9a36-7be8fb280294 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:13.088748 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:13.095942 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a169b53-d3a7-499b-9f48-dc65015cb425 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:13.131545 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a122c55-32ca-4c9e-bf30-b76278750a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:13.132175 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a122c55-32ca-4c9e-bf30-b76278750a54 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:13.132452 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a122c55-32ca-4c9e-bf30-b76278750a54 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:13.132779 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a122c55-32ca-4c9e-bf30-b76278750a54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:13.144131 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a122c55-32ca-4c9e-bf30-b76278750a54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:13.145583 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a122c55-32ca-4c9e-bf30-b76278750a54 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:13.161113 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-3a122c55-32ca-4c9e-bf30-b76278750a54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:55:13.161113 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:13.161113 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:55:13.161113 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:55:13.161113 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:55:13.161113 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:55:13.161113 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:55:13.161113 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:55:13.161113 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:55:13.161113 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:55:13.161113 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:55:13.161113 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:55:13.161113 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:55:13.161113 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:55:13.161113 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:55:13.161113 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:55:13.162093 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:55:13.162093 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:55:13.162093 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:55:13.162093 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:55:13.162093 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:55:13.162093 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:55:13.162093 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:55:13.162093 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:55:13.162093 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:55:13.162093 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:55:13.162093 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:55:13.162093 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:55:13.162093 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:55:13.162093 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:55:13.162093 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:55:13.162093 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:55:13.162093 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:55:13.163013 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:55:13.163013 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:55:13.163013 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:55:13.163013 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 491/970] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:55:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:55:13.169182 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a122c55-32ca-4c9e-bf30-b76278750a54 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:13.178720 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e46f2ce-e9d7-419d-9a36-7be8fb280294 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:13.204323 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:13.215139 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4c2651-70e7-46a4-80fb-cb13af85e2de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:13.215809 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c2651-70e7-46a4-80fb-cb13af85e2de None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:13.216718 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c2651-70e7-46a4-80fb-cb13af85e2de None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:13.216718 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c2651-70e7-46a4-80fb-cb13af85e2de None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:13.217037 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4c2651-70e7-46a4-80fb-cb13af85e2de None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:13.218522 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4c2651-70e7-46a4-80fb-cb13af85e2de None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:13.227532 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-ee4c2651-70e7-46a4-80fb-cb13af85e2de None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 16:55:13.227532 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:13.227532 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 16:55:13.227532 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 16:55:13.227532 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 16:55:13.227532 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 16:55:13.227532 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 16:55:13.227532 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 16:55:13.227532 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 16:55:13.227532 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:55:13.227532 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 16:55:13.227532 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:55:13.227532 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:13.227532 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:55:13.227532 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:55:13.227532 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:55:13.227532 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:55:13.229683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:55:13.229683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:55:13.229683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:55:13.229683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:55:13.229683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:55:13.229683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:55:13.229683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 16:55:13.229683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 16:55:13.229683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 16:55:13.229683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 16:55:13.229683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:55:13.229683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:55:13.229683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:55:13.229683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:55:13.229683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:55:13.229683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:55:13.230519 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:55:13.230519 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:55:13.230519 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:55:13.230519 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:55:13.230519 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:55:13.230519 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:55:13.230519 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:55:13.230519 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:55:13.230519 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:55:13.230519 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:55:13.230519 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 16:55:13.230519 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 16:55:13.230519 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 16:55:13.230519 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:55:13.230519 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 480/971] 10.222.0.82 () {56 vars in 1126 bytes} [Thu Aug 18 16:55:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:55:13.231359 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e921cd17-0a78-49d0-8bf4-482679a22e82 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:13.231359 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e921cd17-0a78-49d0-8bf4-482679a22e82 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:13.231359 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e921cd17-0a78-49d0-8bf4-482679a22e82 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:13.236094 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 481/972] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:55:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:55:13.236094 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e921cd17-0a78-49d0-8bf4-482679a22e82 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:13.279510 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a122c55-32ca-4c9e-bf30-b76278750a54 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:13.314570 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10848f73-ece1-4540-839e-cb1397d1c0d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:13.315211 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10848f73-ece1-4540-839e-cb1397d1c0d6 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:13.315491 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10848f73-ece1-4540-839e-cb1397d1c0d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:13.315764 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10848f73-ece1-4540-839e-cb1397d1c0d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:13.316416 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10848f73-ece1-4540-839e-cb1397d1c0d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:13.316719 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10848f73-ece1-4540-839e-cb1397d1c0d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:13.318553 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10848f73-ece1-4540-839e-cb1397d1c0d6 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:13.327118 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:13.332639 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 492/973] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:55:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:55:13.333846 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451d4eb0-b088-4da1-8388-78a715b00411 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:13.334132 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451d4eb0-b088-4da1-8388-78a715b00411 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:13.334407 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451d4eb0-b088-4da1-8388-78a715b00411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:13.349048 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e921cd17-0a78-49d0-8bf4-482679a22e82 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:13.353678 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-451d4eb0-b088-4da1-8388-78a715b00411 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:55:13.354893 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-451d4eb0-b088-4da1-8388-78a715b00411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:55:13.385144 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccced35-6e67-48ab-9e68-78b1b074ce85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:13.398434 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccced35-6e67-48ab-9e68-78b1b074ce85 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:13.398434 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccced35-6e67-48ab-9e68-78b1b074ce85 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:13.398434 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccced35-6e67-48ab-9e68-78b1b074ce85 None admin] PATH_INFO: `/v3/services` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:13.398434 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccced35-6e67-48ab-9e68-78b1b074ce85 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:13.398434 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccced35-6e67-48ab-9e68-78b1b074ce85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:13.398434 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccced35-6e67-48ab-9e68-78b1b074ce85 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:13.398849 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 482/974] 10.222.0.82 () {56 vars in 1122 bytes} [Thu Aug 18 16:55:13 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:13.398849 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccced35-6e67-48ab-9e68-78b1b074ce85 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:13.427855 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 493/975] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:55:13 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 16:55:13.428770 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451d4eb0-b088-4da1-8388-78a715b00411 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:13.459695 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:13.524059 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccced35-6e67-48ab-9e68-78b1b074ce85 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:13.556118 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451d4eb0-b088-4da1-8388-78a715b00411 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:13.570104 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ded921-a2f7-4c51-8f64-941d074617e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:13.571467 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ded921-a2f7-4c51-8f64-941d074617e2 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:13.572723 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ded921-a2f7-4c51-8f64-941d074617e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:13.573378 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ded921-a2f7-4c51-8f64-941d074617e2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:13.584331 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ded921-a2f7-4c51-8f64-941d074617e2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:13.586415 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ded921-a2f7-4c51-8f64-941d074617e2 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:13.598827 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b778e51a-cb2b-464c-ab94-89df5f661926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:13.600035 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778e51a-cb2b-464c-ab94-89df5f661926 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:13.600576 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778e51a-cb2b-464c-ab94-89df5f661926 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:13.601098 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778e51a-cb2b-464c-ab94-89df5f661926 None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:13.601590 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-29ded921-a2f7-4c51-8f64-941d074617e2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 18 16:55:13.601590 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:13.601590 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:55:13.601590 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:55:13.601590 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:55:13.601590 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:55:13.601590 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:55:13.601590 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:55:13.601590 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:55:13.601590 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:55:13.601590 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:55:13.601590 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:55:13.601590 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 16:55:13.601590 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 16:55:13.601590 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 16:55:13.601590 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 16:55:13.602514 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:55:13.602514 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:55:13.602514 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 16:55:13.602514 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:55:13.602514 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 16:55:13.602514 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:55:13.602514 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:55:13.602514 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:55:13.602514 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:55:13.602514 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:55:13.602514 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:55:13.602514 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:55:13.602514 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:55:13.602514 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:55:13.602514 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:55:13.602514 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:55:13.602514 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:55:13.602514 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:55:13.603786 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:55:13.603786 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:55:13.603786 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 16:55:13.603786 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 16:55:13.603786 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 16:55:13.603786 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 16:55:13.603786 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 16:55:13.603786 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 16:55:13.603786 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 18 16:55:13.603786 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:55:13.604328 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b778e51a-cb2b-464c-ab94-89df5f661926 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:13.604328 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b778e51a-cb2b-464c-ab94-89df5f661926 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:13.604949 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b778e51a-cb2b-464c-ab94-89df5f661926 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:13.610866 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 483/976] 10.222.0.82 () {56 vars in 1117 bytes} [Thu Aug 18 16:55:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:55:13.611951 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ded921-a2f7-4c51-8f64-941d074617e2 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:13.615194 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 494/977] 10.222.0.82 () {56 vars in 1122 bytes} [Thu Aug 18 16:55:13 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:13.616860 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b778e51a-cb2b-464c-ab94-89df5f661926 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:13.624470 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:13.711251 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:13.719844 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ded921-a2f7-4c51-8f64-941d074617e2 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:13.724893 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b778e51a-cb2b-464c-ab94-89df5f661926 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:13.753836 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeabde5-92c2-4178-9be5-2253dee92b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:13.754443 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeabde5-92c2-4178-9be5-2253dee92b63 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:13.754731 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeabde5-92c2-4178-9be5-2253dee92b63 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:13.755041 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeabde5-92c2-4178-9be5-2253dee92b63 None admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:13.755763 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeabde5-92c2-4178-9be5-2253dee92b63 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:13.756068 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeabde5-92c2-4178-9be5-2253dee92b63 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:13.757503 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeabde5-92c2-4178-9be5-2253dee92b63 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:13.764266 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9f59d7-3d54-4f2b-a2ba-28970191fc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:13.765150 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f59d7-3d54-4f2b-a2ba-28970191fc38 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:13.765521 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f59d7-3d54-4f2b-a2ba-28970191fc38 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:13.765906 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f59d7-3d54-4f2b-a2ba-28970191fc38 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:13.766697 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9f59d7-3d54-4f2b-a2ba-28970191fc38 None admin] RBAC: Adding query filter params (interface=admin, service_id=6d5ec5c4407e40758cd24a0aa6d8507f, region_id=RegionOne) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:13.767376 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9f59d7-3d54-4f2b-a2ba-28970191fc38 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:13.768649 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9f59d7-3d54-4f2b-a2ba-28970191fc38 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:13.775028 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 495/978] 10.222.0.82 () {56 vars in 1249 bytes} [Thu Aug 18 16:55:13 2022] GET /identity/v3/endpoints?service_id=6d5ec5c4407e40758cd24a0aa6d8507f&interface=admin®ion_id=RegionOne => generated 193 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:13.775939 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9f59d7-3d54-4f2b-a2ba-28970191fc38 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:13.782490 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 484/979] 10.222.0.82 () {56 vars in 1149 bytes} [Thu Aug 18 16:55:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:13.787891 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeabde5-92c2-4178-9be5-2253dee92b63 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:13.859364 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:13.889131 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9f59d7-3d54-4f2b-a2ba-28970191fc38 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:13.894732 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeabde5-92c2-4178-9be5-2253dee92b63 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:13.928088 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1e6e7b-ebe1-4726-9019-16082c5d8616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:13.928693 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e6e7b-ebe1-4726-9019-16082c5d8616 None admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:13.928970 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e6e7b-ebe1-4726-9019-16082c5d8616 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:13.929240 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e6e7b-ebe1-4726-9019-16082c5d8616 None admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:13.929860 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1e6e7b-ebe1-4726-9019-16082c5d8616 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:13.931734 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed81e12-8eef-4cd7-8115-9450e9fc51d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:13.931948 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1e6e7b-ebe1-4726-9019-16082c5d8616 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:13.932548 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed81e12-8eef-4cd7-8115-9450e9fc51d9 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:13.932927 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed81e12-8eef-4cd7-8115-9450e9fc51d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:13.933299 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed81e12-8eef-4cd7-8115-9450e9fc51d9 None admin] PATH_INFO: `/v3/services` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:13.933995 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed81e12-8eef-4cd7-8115-9450e9fc51d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:13.935489 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed81e12-8eef-4cd7-8115-9450e9fc51d9 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:13.942028 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 485/980] 10.222.0.82 () {56 vars in 1087 bytes} [Thu Aug 18 16:55:13 2022] GET /identity/v3/services => generated 1727 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 16:55:13.943108 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed81e12-8eef-4cd7-8115-9450e9fc51d9 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:13.996981 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.sql.core [None req-be1e6e7b-ebe1-4726-9019-16082c5d8616 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 18 16:55:13.996981 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 18 16:55:13.996981 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [parameters: {'user_id': '9ca71aebb357453a9c894e25cd238990', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 18 16:55:13.996981 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81649) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 18 16:55:14.004242 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-be1e6e7b-ebe1-4726-9019-16082c5d8616 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 18 16:55:14.004242 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:14.004242 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 18 16:55:14.004242 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 18 16:55:14.004242 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 18 16:55:14.004242 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 18 16:55:14.004242 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 18 16:55:14.004242 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application result = self._query(query) Aug 18 16:55:14.004242 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 18 16:55:14.004242 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application conn.query(q) Aug 18 16:55:14.004242 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 18 16:55:14.004242 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 18 16:55:14.004242 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 18 16:55:14.004242 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application result.read() Aug 18 16:55:14.004242 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 18 16:55:14.005388 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 18 16:55:14.005388 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 18 16:55:14.005388 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 18 16:55:14.005388 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 18 16:55:14.005388 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 18 16:55:14.005388 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 18 16:55:14.005388 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 18 16:55:14.005388 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 18 16:55:14.005388 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:55:14.005388 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 18 16:55:14.005388 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:55:14.005388 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:14.005388 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 18 16:55:14.005388 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 18 16:55:14.005388 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 18 16:55:14.005388 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 18 16:55:14.005388 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 18 16:55:14.005388 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application next(self.gen) Aug 18 16:55:14.006554 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 18 16:55:14.006554 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application yield resource Aug 18 16:55:14.006554 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 18 16:55:14.006554 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application next(self.gen) Aug 18 16:55:14.006554 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 18 16:55:14.006554 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self.session.rollback() Aug 18 16:55:14.006554 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 18 16:55:14.006554 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self.force_reraise() Aug 18 16:55:14.006554 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 18 16:55:14.006554 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise self.value Aug 18 16:55:14.006554 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 18 16:55:14.006554 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 18 16:55:14.006554 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 18 16:55:14.006554 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application session.commit() Aug 18 16:55:14.006554 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 18 16:55:14.006554 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 18 16:55:14.006554 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 18 16:55:14.007649 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self._prepare_impl() Aug 18 16:55:14.007649 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 18 16:55:14.007649 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self.session.flush() Aug 18 16:55:14.007649 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 18 16:55:14.007649 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self._flush(objects) Aug 18 16:55:14.007649 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 18 16:55:14.007649 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 18 16:55:14.007649 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 18 16:55:14.007649 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application compat.raise_( Aug 18 16:55:14.007649 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 18 16:55:14.007649 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception Aug 18 16:55:14.007649 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 18 16:55:14.007649 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application flush_context.execute() Aug 18 16:55:14.007649 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 18 16:55:14.007649 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rec.execute(self) Aug 18 16:55:14.007649 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 18 16:55:14.007649 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 18 16:55:14.008725 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 18 16:55:14.008725 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 18 16:55:14.008725 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 18 16:55:14.008725 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 18 16:55:14.008725 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 18 16:55:14.008725 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 18 16:55:14.008725 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 18 16:55:14.008725 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 18 16:55:14.008725 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 18 16:55:14.008725 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 18 16:55:14.008725 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 18 16:55:14.008725 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 18 16:55:14.008725 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 18 16:55:14.008725 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 18 16:55:14.008725 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 18 16:55:14.008725 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception Aug 18 16:55:14.010816 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 18 16:55:14.010816 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 18 16:55:14.010816 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 18 16:55:14.010816 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 18 16:55:14.010816 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 18 16:55:14.010816 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application result = self._query(query) Aug 18 16:55:14.010816 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 18 16:55:14.010816 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application conn.query(q) Aug 18 16:55:14.010816 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 18 16:55:14.010816 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 18 16:55:14.010816 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 18 16:55:14.010816 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application result.read() Aug 18 16:55:14.010816 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 18 16:55:14.010816 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 18 16:55:14.010816 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 18 16:55:14.010816 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 18 16:55:14.011948 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 18 16:55:14.011948 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 18 16:55:14.011948 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 18 16:55:14.011948 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 18 16:55:14.011948 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 18 16:55:14.011948 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 18 16:55:14.011948 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [parameters: {'user_id': '9ca71aebb357453a9c894e25cd238990', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 18 16:55:14.011948 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 18 16:55:14.011948 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:55:14.011948 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 16:55:14.011948 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:55:14.011948 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:14.011948 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:55:14.011948 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:55:14.011948 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:55:14.012943 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:55:14.012943 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:55:14.012943 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:55:14.012943 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:55:14.012943 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:55:14.012943 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:55:14.012943 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:55:14.012943 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 18 16:55:14.012943 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 18 16:55:14.012943 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:55:14.012943 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:55:14.012943 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 16:55:14.012943 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:55:14.012943 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 16:55:14.012943 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:55:14.012943 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 18 16:55:14.012943 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 18 16:55:14.013952 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 18 16:55:14.013952 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 18 16:55:14.013952 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 18 16:55:14.013952 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 18 16:55:14.013952 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 18 16:55:14.013952 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:55:14.018980 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 496/981] 10.222.0.82 () {60 vars in 1131 bytes} [Thu Aug 18 16:55:13 2022] POST /identity/v3/users => generated 160 bytes in 243 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:55:14.020746 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1e6e7b-ebe1-4726-9019-16082c5d8616 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:14.040019 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:14.094656 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed81e12-8eef-4cd7-8115-9450e9fc51d9 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:14.131282 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:14.140085 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca24723-4be2-47df-96dd-ba4eade854b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:14.143064 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1e6e7b-ebe1-4726-9019-16082c5d8616 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:14.143177 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca24723-4be2-47df-96dd-ba4eade854b4 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:14.143177 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca24723-4be2-47df-96dd-ba4eade854b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:14.143177 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca24723-4be2-47df-96dd-ba4eade854b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:14.183334 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1292b6-6334-4415-b20b-ed3682ebd033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:14.183978 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1292b6-6334-4415-b20b-ed3682ebd033 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:14.184256 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1292b6-6334-4415-b20b-ed3682ebd033 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:14.184528 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1292b6-6334-4415-b20b-ed3682ebd033 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:14.192276 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1292b6-6334-4415-b20b-ed3682ebd033 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:14.193913 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1292b6-6334-4415-b20b-ed3682ebd033 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:14.206318 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-1c1292b6-6334-4415-b20b-ed3682ebd033 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 18 16:55:14.206318 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:14.206318 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:55:14.206318 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:55:14.206318 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:55:14.206318 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:55:14.206318 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:55:14.206318 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:55:14.206318 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:55:14.206318 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:55:14.206318 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:55:14.206318 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:55:14.206318 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 16:55:14.206318 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 16:55:14.206318 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 16:55:14.206318 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 16:55:14.207353 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:55:14.207353 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:55:14.207353 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 16:55:14.207353 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:55:14.207353 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 16:55:14.207353 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:55:14.207353 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:55:14.207353 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:55:14.207353 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:55:14.207353 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:55:14.207353 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:55:14.207353 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:55:14.207353 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:55:14.207353 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:55:14.207353 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:55:14.207353 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:55:14.207353 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:55:14.207353 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:55:14.208307 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:55:14.208307 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:55:14.208307 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 16:55:14.208307 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 16:55:14.208307 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 16:55:14.208307 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 16:55:14.208307 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 16:55:14.208307 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 16:55:14.208307 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 18 16:55:14.208307 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:55:14.208307 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 497/982] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:55:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:55:14.216263 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1292b6-6334-4415-b20b-ed3682ebd033 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:14.226730 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca24723-4be2-47df-96dd-ba4eade854b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:14.228465 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca24723-4be2-47df-96dd-ba4eade854b4 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:14.295801 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:14.321164 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1292b6-6334-4415-b20b-ed3682ebd033 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:14.339573 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 486/983] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:55:13 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 16:55:14.345136 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca24723-4be2-47df-96dd-ba4eade854b4 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:14.365495 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184821a0-5a46-4afa-b127-2358d6a97072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:14.366346 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184821a0-5a46-4afa-b127-2358d6a97072 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:14.366796 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184821a0-5a46-4afa-b127-2358d6a97072 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:14.367255 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184821a0-5a46-4afa-b127-2358d6a97072 None admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:14.368093 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184821a0-5a46-4afa-b127-2358d6a97072 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:14.369040 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184821a0-5a46-4afa-b127-2358d6a97072 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:14.371039 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184821a0-5a46-4afa-b127-2358d6a97072 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:14.377998 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:14.422942 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 498/984] 10.222.0.82 () {56 vars in 1101 bytes} [Thu Aug 18 16:55:14 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:55:14.428162 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b7481a-577d-4e61-b497-3cc4dda3eb08 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:14.428592 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b7481a-577d-4e61-b497-3cc4dda3eb08 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:14.428715 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b7481a-577d-4e61-b497-3cc4dda3eb08 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:14.430104 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 499/985] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:55:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:55:14.436392 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df203881-1e34-48bb-87f1-c16b4c92a243 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:14.436722 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df203881-1e34-48bb-87f1-c16b4c92a243 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:14.436962 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df203881-1e34-48bb-87f1-c16b4c92a243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:14.460564 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-df203881-1e34-48bb-87f1-c16b4c92a243 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:55:14.462203 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-df203881-1e34-48bb-87f1-c16b4c92a243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:55:14.467205 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca24723-4be2-47df-96dd-ba4eade854b4 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:14.519977 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeb4164-d862-47ab-b0e4-0a601698915b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:14.521285 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb4164-d862-47ab-b0e4-0a601698915b None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:14.521786 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb4164-d862-47ab-b0e4-0a601698915b None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:14.522396 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb4164-d862-47ab-b0e4-0a601698915b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:14.537314 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeb4164-d862-47ab-b0e4-0a601698915b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:14.539298 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeb4164-d862-47ab-b0e4-0a601698915b None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:14.546270 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 500/986] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:55:14 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 16:55:14.557582 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-bdeb4164-d862-47ab-b0e4-0a601698915b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:55:14.557582 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:14.557582 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:55:14.557582 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:55:14.557582 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:55:14.557582 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:55:14.557582 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:55:14.557582 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:55:14.557582 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:55:14.557582 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:55:14.557582 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:55:14.557582 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:55:14.557582 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:55:14.557582 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:55:14.557582 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:55:14.557582 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:55:14.558622 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:55:14.558622 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:55:14.558622 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:55:14.558622 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:55:14.558622 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:55:14.558622 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:55:14.558622 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:55:14.558622 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:55:14.558622 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:55:14.558622 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:55:14.558622 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:55:14.558622 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:55:14.558622 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:55:14.558622 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:55:14.558622 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:55:14.558622 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:55:14.558622 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:55:14.559505 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:55:14.559505 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:55:14.559505 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:55:14.561680 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 487/987] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:55:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 16:55:14.567103 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df203881-1e34-48bb-87f1-c16b4c92a243 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:14.634860 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:14.672036 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df203881-1e34-48bb-87f1-c16b4c92a243 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:14.708449 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af678756-8931-4ae4-8049-70d885bfc227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:14.709378 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af678756-8931-4ae4-8049-70d885bfc227 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:14.709812 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af678756-8931-4ae4-8049-70d885bfc227 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:14.710229 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af678756-8931-4ae4-8049-70d885bfc227 None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:14.711062 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af678756-8931-4ae4-8049-70d885bfc227 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:14.711505 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af678756-8931-4ae4-8049-70d885bfc227 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:14.713312 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af678756-8931-4ae4-8049-70d885bfc227 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:14.725241 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 501/988] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:55:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:14.731489 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeb4164-d862-47ab-b0e4-0a601698915b None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:14.823841 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af678756-8931-4ae4-8049-70d885bfc227 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:14.824327 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:14.844065 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeb4164-d862-47ab-b0e4-0a601698915b None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:14.885142 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd70b9dc-a844-40a1-8818-c8857706ec9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:14.885930 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd70b9dc-a844-40a1-8818-c8857706ec9d None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:14.886321 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd70b9dc-a844-40a1-8818-c8857706ec9d None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:14.886691 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd70b9dc-a844-40a1-8818-c8857706ec9d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:14.894099 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd70b9dc-a844-40a1-8818-c8857706ec9d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:14.895700 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd70b9dc-a844-40a1-8818-c8857706ec9d None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:14.907189 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-bd70b9dc-a844-40a1-8818-c8857706ec9d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:55:14.907189 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:14.907189 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:55:14.907189 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:55:14.907189 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:55:14.907189 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:55:14.907189 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:55:14.907189 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:55:14.907189 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:55:14.907189 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:55:14.907189 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:55:14.907189 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:55:14.907189 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 16:55:14.907189 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 16:55:14.907189 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 16:55:14.908285 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 16:55:14.908285 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:55:14.908285 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:55:14.908285 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:55:14.908285 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:55:14.908285 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:55:14.908285 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:55:14.908285 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:55:14.908285 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:55:14.908285 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:55:14.908285 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:55:14.908285 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:55:14.908285 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:55:14.908285 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:55:14.908285 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:55:14.908285 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:55:14.908285 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:55:14.909349 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 16:55:14.909349 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 16:55:14.909349 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 16:55:14.909349 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 16:55:14.909349 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 16:55:14.909349 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 16:55:14.909349 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:55:14.909349 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:55:14.910452 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 488/989] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 16:55:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:55:14.916398 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd70b9dc-a844-40a1-8818-c8857706ec9d None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:14.931766 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af678756-8931-4ae4-8049-70d885bfc227 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:14.955025 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:14.965887 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcccd665-d99c-4c55-aed0-993ba4ac4d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:14.966519 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcccd665-d99c-4c55-aed0-993ba4ac4d30 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:14.966803 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcccd665-d99c-4c55-aed0-993ba4ac4d30 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:14.975916 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcccd665-d99c-4c55-aed0-993ba4ac4d30 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:14.975916 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcccd665-d99c-4c55-aed0-993ba4ac4d30 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:14.975916 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcccd665-d99c-4c55-aed0-993ba4ac4d30 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:14.978292 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-fcccd665-d99c-4c55-aed0-993ba4ac4d30 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 18 16:55:14.978292 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:14.978292 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 16:55:14.978292 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 16:55:14.978292 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 16:55:14.978292 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 16:55:14.978292 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 16:55:14.978292 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 16:55:14.978292 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 18 16:55:14.978292 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:55:14.978292 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 16:55:14.978292 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:55:14.978292 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:14.978292 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:55:14.978292 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:55:14.978292 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:55:14.978292 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:55:14.979303 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:55:14.979303 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:55:14.979303 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:55:14.979303 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:55:14.979303 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:55:14.979303 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:55:14.979303 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 16:55:14.979303 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 16:55:14.979303 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 16:55:14.979303 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 16:55:14.979303 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:55:14.979303 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:55:14.979303 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:55:14.979303 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:55:14.979303 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:55:14.979303 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:55:14.980130 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:55:14.980130 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:55:14.980130 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:55:14.980130 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:55:14.980130 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:55:14.980130 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:55:14.980130 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:55:14.980130 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:55:14.980130 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:55:14.980130 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:55:14.980130 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 16:55:14.980130 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 16:55:14.980130 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 18 16:55:14.980130 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:55:14.984654 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 502/990] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 16:55:14 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 16:55:14.985289 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcccd665-d99c-4c55-aed0-993ba4ac4d30 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:15.025194 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd70b9dc-a844-40a1-8818-c8857706ec9d None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:15.061175 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45078db-e48a-4a68-a3a7-9c798e31d4e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:15.061848 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45078db-e48a-4a68-a3a7-9c798e31d4e3 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:15.062134 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45078db-e48a-4a68-a3a7-9c798e31d4e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:15.062413 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45078db-e48a-4a68-a3a7-9c798e31d4e3 None admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:15.063190 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45078db-e48a-4a68-a3a7-9c798e31d4e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:15.063503 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45078db-e48a-4a68-a3a7-9c798e31d4e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:15.065019 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45078db-e48a-4a68-a3a7-9c798e31d4e3 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:15.082373 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 489/991] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 16:55:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:15.083712 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45078db-e48a-4a68-a3a7-9c798e31d4e3 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:15.087944 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:15.092914 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcccd665-d99c-4c55-aed0-993ba4ac4d30 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:15.131731 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed7ace7-c92b-43dd-9bc1-ea72b2fe0378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:15.132486 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed7ace7-c92b-43dd-9bc1-ea72b2fe0378 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:15.132864 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed7ace7-c92b-43dd-9bc1-ea72b2fe0378 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:15.133233 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed7ace7-c92b-43dd-9bc1-ea72b2fe0378 None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:15.133980 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed7ace7-c92b-43dd-9bc1-ea72b2fe0378 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:15.135293 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed7ace7-c92b-43dd-9bc1-ea72b2fe0378 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:15.136672 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed7ace7-c92b-43dd-9bc1-ea72b2fe0378 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:15.141723 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 503/992] 10.222.0.82 () {56 vars in 1112 bytes} [Thu Aug 18 16:55:14 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:55:15.147298 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed7ace7-c92b-43dd-9bc1-ea72b2fe0378 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:15.200581 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:15.203135 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45078db-e48a-4a68-a3a7-9c798e31d4e3 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:15.239902 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6023b122-351a-43fb-ab94-fabc6091e15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:15.240508 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6023b122-351a-43fb-ab94-fabc6091e15b None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:15.240789 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6023b122-351a-43fb-ab94-fabc6091e15b None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:15.241062 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6023b122-351a-43fb-ab94-fabc6091e15b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:15.241896 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6023b122-351a-43fb-ab94-fabc6091e15b None admin] RBAC: Adding query filter params (role.id=bae4160aa9de4d69b42641fd18c53e55, scope.project.id=2293c0aab61f44849d0d48b214cde7ea, user.id=5de19a54ed104bcdb35ee4e5e793f1e7) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:15.242209 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6023b122-351a-43fb-ab94-fabc6091e15b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:15.244217 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6023b122-351a-43fb-ab94-fabc6091e15b None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:15.249849 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 490/993] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 16:55:15 2022] GET /identity/v3/role_assignments?user.id=5de19a54ed104bcdb35ee4e5e793f1e7&scope.project.id=2293c0aab61f44849d0d48b214cde7ea&role.id=bae4160aa9de4d69b42641fd18c53e55 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:55:15.269495 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed7ace7-c92b-43dd-9bc1-ea72b2fe0378 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:15.309974 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521420da-ccbd-4b3d-b5ff-82df726cb106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:15.310697 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521420da-ccbd-4b3d-b5ff-82df726cb106 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:15.311007 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521420da-ccbd-4b3d-b5ff-82df726cb106 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:15.311277 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521420da-ccbd-4b3d-b5ff-82df726cb106 None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:15.311923 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521420da-ccbd-4b3d-b5ff-82df726cb106 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:15.312220 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521420da-ccbd-4b3d-b5ff-82df726cb106 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:15.313601 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521420da-ccbd-4b3d-b5ff-82df726cb106 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:15.324172 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 504/994] 10.222.0.82 () {56 vars in 1112 bytes} [Thu Aug 18 16:55:15 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:15.325333 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6023b122-351a-43fb-ab94-fabc6091e15b None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:15.397584 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:15.436890 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6023b122-351a-43fb-ab94-fabc6091e15b None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:15.472704 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2aa1f8-4a96-4bfa-9e28-565402a4aa9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:15.473312 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2aa1f8-4a96-4bfa-9e28-565402a4aa9e None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:15.473589 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2aa1f8-4a96-4bfa-9e28-565402a4aa9e None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:15.473859 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2aa1f8-4a96-4bfa-9e28-565402a4aa9e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:15.474543 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2aa1f8-4a96-4bfa-9e28-565402a4aa9e None admin] RBAC: Adding query filter params (interface=public, service_id=f3095e941b6344ea80290524cc31cd44, region_id=RegionOne) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:15.474848 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2aa1f8-4a96-4bfa-9e28-565402a4aa9e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:15.476253 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2aa1f8-4a96-4bfa-9e28-565402a4aa9e None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:15.486970 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 491/995] 10.222.0.82 () {56 vars in 1251 bytes} [Thu Aug 18 16:55:15 2022] GET /identity/v3/endpoints?service_id=f3095e941b6344ea80290524cc31cd44&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:55:15.488109 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521420da-ccbd-4b3d-b5ff-82df726cb106 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:15.591573 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521420da-ccbd-4b3d-b5ff-82df726cb106 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:15.606387 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:15.627173 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc90140-cfac-485c-94e1-9443504cb0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:15.627788 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc90140-cfac-485c-94e1-9443504cb0f4 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:15.628068 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc90140-cfac-485c-94e1-9443504cb0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:15.628400 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc90140-cfac-485c-94e1-9443504cb0f4 None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:15.629005 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc90140-cfac-485c-94e1-9443504cb0f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:15.630365 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc90140-cfac-485c-94e1-9443504cb0f4 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:15.642986 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 505/996] 10.222.0.82 () {56 vars in 1087 bytes} [Thu Aug 18 16:55:15 2022] GET /identity/v3/services => generated 1727 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 16:55:15.953724 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d14ea-616c-444d-beed-270a6b3ef9e5 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:15.954235 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d14ea-616c-444d-beed-270a6b3ef9e5 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:15.954235 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d14ea-616c-444d-beed-270a6b3ef9e5 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:15.955760 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 492/997] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:55:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:55:15.960900 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e856bffd-551c-4dfa-9413-30486503aa54 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:15.961193 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e856bffd-551c-4dfa-9413-30486503aa54 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:15.961467 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e856bffd-551c-4dfa-9413-30486503aa54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:15.981245 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-e856bffd-551c-4dfa-9413-30486503aa54 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:55:15.982422 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-e856bffd-551c-4dfa-9413-30486503aa54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:55:16.048271 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 506/998] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:55:15 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 16:55:16.314048 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0d14ea-616c-444d-beed-270a6b3ef9e5 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:16.418769 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0d14ea-616c-444d-beed-270a6b3ef9e5 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:16.449748 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:16.455279 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fbadc2-eea6-4fa9-8c4e-ae6dd3a5c82c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:16.455906 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fbadc2-eea6-4fa9-8c4e-ae6dd3a5c82c None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:16.456182 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fbadc2-eea6-4fa9-8c4e-ae6dd3a5c82c None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:16.456448 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fbadc2-eea6-4fa9-8c4e-ae6dd3a5c82c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:16.457012 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fbadc2-eea6-4fa9-8c4e-ae6dd3a5c82c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:16.458379 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fbadc2-eea6-4fa9-8c4e-ae6dd3a5c82c None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:16.467429 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-97fbadc2-eea6-4fa9-8c4e-ae6dd3a5c82c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 16:55:16.467429 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:16.467429 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 16:55:16.467429 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 16:55:16.467429 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 16:55:16.467429 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 16:55:16.467429 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 16:55:16.467429 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 16:55:16.467429 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 16:55:16.467429 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:55:16.467429 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 16:55:16.467429 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:55:16.467429 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:16.467429 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:55:16.467429 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:55:16.467429 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:55:16.467429 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:55:16.478874 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:55:16.478874 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:55:16.478874 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:55:16.478874 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:55:16.478874 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:55:16.478874 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:55:16.478874 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 16:55:16.478874 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 16:55:16.478874 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 16:55:16.478874 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 16:55:16.478874 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:55:16.478874 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:55:16.478874 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:55:16.478874 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:55:16.478874 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:55:16.478874 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:55:16.480010 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:55:16.480010 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:55:16.480010 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:55:16.480010 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:55:16.480010 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:55:16.480010 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:55:16.480010 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:55:16.480010 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:55:16.480010 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:55:16.480010 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:55:16.480010 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 16:55:16.480010 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 16:55:16.480010 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 16:55:16.480010 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:55:16.480010 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 493/999] 10.222.0.82 () {56 vars in 1126 bytes} [Thu Aug 18 16:55:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:55:16.480787 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e856bffd-551c-4dfa-9413-30486503aa54 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:16.577081 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e856bffd-551c-4dfa-9413-30486503aa54 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:16.586151 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:16.611041 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f514773-b504-446d-bc12-945d13a6a99d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:16.611653 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f514773-b504-446d-bc12-945d13a6a99d None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:16.611932 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f514773-b504-446d-bc12-945d13a6a99d None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:16.612256 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f514773-b504-446d-bc12-945d13a6a99d None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:16.612817 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f514773-b504-446d-bc12-945d13a6a99d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:16.613114 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f514773-b504-446d-bc12-945d13a6a99d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:16.614596 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f514773-b504-446d-bc12-945d13a6a99d None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:16.620129 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 507/1000] 10.222.0.82 () {56 vars in 1122 bytes} [Thu Aug 18 16:55:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:16.625765 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fbadc2-eea6-4fa9-8c4e-ae6dd3a5c82c None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:16.727196 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fbadc2-eea6-4fa9-8c4e-ae6dd3a5c82c None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:16.761086 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b027039-295b-4bc0-a0e7-839cfb4c5ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:16.761692 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b027039-295b-4bc0-a0e7-839cfb4c5ef1 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:16.761965 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b027039-295b-4bc0-a0e7-839cfb4c5ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:16.766928 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b027039-295b-4bc0-a0e7-839cfb4c5ef1 None admin] PATH_INFO: `/v3/services` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:16.766928 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b027039-295b-4bc0-a0e7-839cfb4c5ef1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:16.766928 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b027039-295b-4bc0-a0e7-839cfb4c5ef1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:16.766928 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b027039-295b-4bc0-a0e7-839cfb4c5ef1 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:16.771940 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 494/1001] 10.222.0.82 () {56 vars in 1122 bytes} [Thu Aug 18 16:55:16 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:16.778115 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f514773-b504-446d-bc12-945d13a6a99d None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:16.883532 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f514773-b504-446d-bc12-945d13a6a99d None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:16.918300 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5969adc-7574-43bd-9a31-c0644939d9ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:16.918925 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5969adc-7574-43bd-9a31-c0644939d9ca None admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:16.919290 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5969adc-7574-43bd-9a31-c0644939d9ca None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:16.919568 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5969adc-7574-43bd-9a31-c0644939d9ca None admin] PATH_INFO: `/v3/endpoints` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:16.920239 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5969adc-7574-43bd-9a31-c0644939d9ca None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:16.921610 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5969adc-7574-43bd-9a31-c0644939d9ca None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:16.999687 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 508/1002] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 16:55:16 2022] POST /identity/v3/endpoints => generated 325 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 16:55:17.237744 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29654d3e-28c0-454b-b159-826345c7c8cd None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:17.238266 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29654d3e-28c0-454b-b159-826345c7c8cd None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:17.238266 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29654d3e-28c0-454b-b159-826345c7c8cd None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:17.239745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 495/1003] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:55:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:55:17.244885 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe035c35-503f-4887-aa9b-324e49c67925 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:17.245245 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe035c35-503f-4887-aa9b-324e49c67925 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:17.245589 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe035c35-503f-4887-aa9b-324e49c67925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:17.264165 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-fe035c35-503f-4887-aa9b-324e49c67925 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:55:17.265346 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-fe035c35-503f-4887-aa9b-324e49c67925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:55:17.292093 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:17.340994 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 509/1004] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:55:17 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 16:55:17.559749 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ef68c-32db-4e0a-a021-dabeb33d9984 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:17.560105 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ef68c-32db-4e0a-a021-dabeb33d9984 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:17.560308 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ef68c-32db-4e0a-a021-dabeb33d9984 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:17.561802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 496/1005] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:55:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:55:17.566716 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e6d3cc-1695-4a89-bc3d-20923698445f None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:17.567167 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e6d3cc-1695-4a89-bc3d-20923698445f None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:17.567325 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e6d3cc-1695-4a89-bc3d-20923698445f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:17.584971 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-c5e6d3cc-1695-4a89-bc3d-20923698445f None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:55:17.586144 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-c5e6d3cc-1695-4a89-bc3d-20923698445f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:55:17.610147 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2ef68c-32db-4e0a-a021-dabeb33d9984 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:17.658027 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 510/1006] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:55:17 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 16:55:17.671982 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:17.723367 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2ef68c-32db-4e0a-a021-dabeb33d9984 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:17.777567 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b23ca1e-8428-468a-89e0-e910567916b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:17.778198 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b23ca1e-8428-468a-89e0-e910567916b6 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:17.778478 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b23ca1e-8428-468a-89e0-e910567916b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:17.778747 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b23ca1e-8428-468a-89e0-e910567916b6 None admin] PATH_INFO: `/v3/services/image` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:17.782035 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b23ca1e-8428-468a-89e0-e910567916b6 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:17.782035 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b23ca1e-8428-468a-89e0-e910567916b6 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:17.791744 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-4b23ca1e-8428-468a-89e0-e910567916b6 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 18 16:55:17.791744 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:17.791744 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 16:55:17.791744 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 16:55:17.791744 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 16:55:17.791744 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 16:55:17.791744 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 16:55:17.791744 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 16:55:17.791744 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 18 16:55:17.791744 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:55:17.791744 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 16:55:17.791744 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:55:17.791744 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:17.791744 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:55:17.791744 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:55:17.791744 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:55:17.791744 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:55:17.792695 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:55:17.792695 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:55:17.792695 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:55:17.792695 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:55:17.792695 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:55:17.792695 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:55:17.792695 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 16:55:17.792695 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 16:55:17.792695 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 16:55:17.792695 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 16:55:17.792695 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:55:17.792695 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:55:17.792695 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:55:17.792695 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:55:17.792695 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:55:17.792695 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:55:17.795865 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:55:17.795865 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:55:17.795865 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:55:17.795865 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:55:17.795865 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:55:17.795865 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:55:17.795865 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:55:17.795865 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:55:17.795865 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:55:17.795865 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:55:17.795865 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 16:55:17.795865 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 16:55:17.795865 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 18 16:55:17.795865 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:55:17.795865 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 497/1007] 10.222.0.82 () {56 vars in 1105 bytes} [Thu Aug 18 16:55:17 2022] GET /identity/v3/services/image => generated 86 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 16:55:17.799135 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e6d3cc-1695-4a89-bc3d-20923698445f None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:17.886842 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:17.898827 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e6d3cc-1695-4a89-bc3d-20923698445f None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:17.932850 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e009549-532d-4520-bca2-72e6eb3b5540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:17.933454 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e009549-532d-4520-bca2-72e6eb3b5540 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:17.933736 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e009549-532d-4520-bca2-72e6eb3b5540 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:17.934002 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e009549-532d-4520-bca2-72e6eb3b5540 None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:17.934617 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e009549-532d-4520-bca2-72e6eb3b5540 None admin] RBAC: Adding query filter params (name=image) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:17.934940 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e009549-532d-4520-bca2-72e6eb3b5540 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:17.936299 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e009549-532d-4520-bca2-72e6eb3b5540 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:17.941110 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 511/1008] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:55:17 2022] GET /identity/v3/services?name=image => generated 122 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:17.947263 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b23ca1e-8428-468a-89e0-e910567916b6 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:17.991522 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e009549-532d-4520-bca2-72e6eb3b5540 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:18.057669 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b23ca1e-8428-468a-89e0-e910567916b6 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:18.081746 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:18.096091 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b6207a-ea86-461e-a806-b657537de1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:18.097409 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b6207a-ea86-461e-a806-b657537de1fb None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:18.098110 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b6207a-ea86-461e-a806-b657537de1fb None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:18.098537 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b6207a-ea86-461e-a806-b657537de1fb None admin] PATH_INFO: `/v3/services` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:18.099611 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b6207a-ea86-461e-a806-b657537de1fb None admin] RBAC: Adding query filter params (type=image) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:18.100580 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b6207a-ea86-461e-a806-b657537de1fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:18.102367 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b6207a-ea86-461e-a806-b657537de1fb None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:18.103584 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e009549-532d-4520-bca2-72e6eb3b5540 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:18.109340 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 498/1009] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:55:17 2022] GET /identity/v3/services?type=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:18.142253 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99f4f08-5857-45cb-bd49-d99d342391b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:18.142880 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f4f08-5857-45cb-bd49-d99d342391b1 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:18.143208 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f4f08-5857-45cb-bd49-d99d342391b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:18.143477 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f4f08-5857-45cb-bd49-d99d342391b1 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:18.144061 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99f4f08-5857-45cb-bd49-d99d342391b1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:18.145467 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99f4f08-5857-45cb-bd49-d99d342391b1 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:18.155853 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-e99f4f08-5857-45cb-bd49-d99d342391b1 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 18 16:55:18.155853 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:18.155853 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 16:55:18.155853 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 16:55:18.155853 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 16:55:18.155853 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 16:55:18.155853 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 16:55:18.155853 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 16:55:18.155853 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 18 16:55:18.155853 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:55:18.155853 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 16:55:18.155853 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:55:18.155853 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:18.155853 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:55:18.155853 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:55:18.155853 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:55:18.155853 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:55:18.156911 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:55:18.156911 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:55:18.156911 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:55:18.156911 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:55:18.156911 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:55:18.156911 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:55:18.156911 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 16:55:18.156911 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 16:55:18.156911 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 16:55:18.156911 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 16:55:18.156911 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:55:18.156911 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:55:18.156911 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:55:18.156911 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:55:18.156911 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:55:18.156911 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:55:18.157746 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:55:18.157746 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:55:18.157746 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:55:18.157746 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:55:18.157746 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:55:18.157746 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:55:18.157746 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:55:18.157746 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:55:18.157746 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:55:18.157746 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:55:18.157746 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 16:55:18.157746 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 16:55:18.157746 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 18 16:55:18.157746 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:55:18.159376 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 512/1010] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 16:55:17 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 16:55:18.163581 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b6207a-ea86-461e-a806-b657537de1fb None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:18.276861 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b6207a-ea86-461e-a806-b657537de1fb None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:18.310405 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe205f52-e11a-4c81-be8f-861e3e34c042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:18.311049 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe205f52-e11a-4c81-be8f-861e3e34c042 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:18.311335 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe205f52-e11a-4c81-be8f-861e3e34c042 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:18.311616 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe205f52-e11a-4c81-be8f-861e3e34c042 None admin] PATH_INFO: `/v3/services` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:18.312214 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe205f52-e11a-4c81-be8f-861e3e34c042 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:18.312506 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe205f52-e11a-4c81-be8f-861e3e34c042 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:18.313870 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe205f52-e11a-4c81-be8f-861e3e34c042 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:18.318837 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 499/1011] 10.222.0.82 () {56 vars in 1112 bytes} [Thu Aug 18 16:55:18 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:18.324889 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99f4f08-5857-45cb-bd49-d99d342391b1 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:18.344073 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:18.449951 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99f4f08-5857-45cb-bd49-d99d342391b1 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:18.487063 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcad0aa-53b4-44bf-8f47-8e5dcff6c273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:18.487676 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcad0aa-53b4-44bf-8f47-8e5dcff6c273 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:18.487954 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcad0aa-53b4-44bf-8f47-8e5dcff6c273 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:18.488231 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcad0aa-53b4-44bf-8f47-8e5dcff6c273 None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:18.488865 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcad0aa-53b4-44bf-8f47-8e5dcff6c273 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:18.489227 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcad0aa-53b4-44bf-8f47-8e5dcff6c273 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:18.490535 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcad0aa-53b4-44bf-8f47-8e5dcff6c273 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:18.501012 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 513/1012] 10.222.0.82 () {56 vars in 1112 bytes} [Thu Aug 18 16:55:18 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:18.502244 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe205f52-e11a-4c81-be8f-861e3e34c042 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:18.558676 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:18.613406 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe205f52-e11a-4c81-be8f-861e3e34c042 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:18.650452 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a400007f-56d6-410a-a07e-cb07b08a12b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:18.651194 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a400007f-56d6-410a-a07e-cb07b08a12b8 None admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:18.651479 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a400007f-56d6-410a-a07e-cb07b08a12b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:18.651751 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a400007f-56d6-410a-a07e-cb07b08a12b8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:18.652403 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a400007f-56d6-410a-a07e-cb07b08a12b8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:18.653800 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a400007f-56d6-410a-a07e-cb07b08a12b8 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:18.739023 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 500/1013] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 16:55:18 2022] POST /identity/v3/endpoints => generated 334 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 16:55:18.955754 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ea203-4afa-46a2-b139-32d03b4aa2ee None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:18.956438 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ea203-4afa-46a2-b139-32d03b4aa2ee None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:18.956438 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ea203-4afa-46a2-b139-32d03b4aa2ee None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:18.957658 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 514/1014] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:55:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:55:18.963692 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f6baa8-45d5-4530-a563-7b21dce87385 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:18.964056 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f6baa8-45d5-4530-a563-7b21dce87385 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:18.964309 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f6baa8-45d5-4530-a563-7b21dce87385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:18.982652 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-b7f6baa8-45d5-4530-a563-7b21dce87385 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:55:18.984037 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-b7f6baa8-45d5-4530-a563-7b21dce87385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:55:19.069968 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 501/1015] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:55:18 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 16:55:19.351511 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6ea203-4afa-46a2-b139-32d03b4aa2ee None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:19.436483 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:19.456639 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6ea203-4afa-46a2-b139-32d03b4aa2ee None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:19.491518 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ff9ca-2593-43a8-9a87-49a1d733ade5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:19.492151 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ff9ca-2593-43a8-9a87-49a1d733ade5 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:19.492432 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ff9ca-2593-43a8-9a87-49a1d733ade5 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:19.492706 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ff9ca-2593-43a8-9a87-49a1d733ade5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:19.494482 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0ff9ca-2593-43a8-9a87-49a1d733ade5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:19.495944 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0ff9ca-2593-43a8-9a87-49a1d733ade5 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:19.497742 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 515/1016] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:55:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:55:19.503412 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f6baa8-45d5-4530-a563-7b21dce87385 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:19.534641 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:19.608063 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f6baa8-45d5-4530-a563-7b21dce87385 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:19.643035 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4c2e94-88d4-4a7c-a11e-99bf49eaa30c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:19.643897 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4c2e94-88d4-4a7c-a11e-99bf49eaa30c None admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:19.644473 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4c2e94-88d4-4a7c-a11e-99bf49eaa30c None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:19.644945 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4c2e94-88d4-4a7c-a11e-99bf49eaa30c None admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:19.645757 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4c2e94-88d4-4a7c-a11e-99bf49eaa30c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:19.647461 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4c2e94-88d4-4a7c-a11e-99bf49eaa30c None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:19.730597 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 502/1017] 10.222.0.82 () {60 vars in 1141 bytes} [Thu Aug 18 16:55:19 2022] POST /identity/v3/projects => generated 311 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 16:55:20.191720 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76d4900-a0af-49a9-a157-731936417f5d None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:20.192230 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76d4900-a0af-49a9-a157-731936417f5d None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:20.192230 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76d4900-a0af-49a9-a157-731936417f5d None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:20.193585 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 516/1018] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:55:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:55:20.199722 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182469ac-1939-43bf-b889-d0167272d122 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:20.200019 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182469ac-1939-43bf-b889-d0167272d122 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:20.200296 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182469ac-1939-43bf-b889-d0167272d122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:20.219302 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-182469ac-1939-43bf-b889-d0167272d122 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:55:20.220527 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-182469ac-1939-43bf-b889-d0167272d122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:55:20.290875 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 503/1019] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:55:20 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 16:55:20.561711 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76d4900-a0af-49a9-a157-731936417f5d None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:20.664120 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76d4900-a0af-49a9-a157-731936417f5d None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:20.698022 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9706bc6c-fff4-4e3b-a46c-fa4e09099cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:20.698628 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706bc6c-fff4-4e3b-a46c-fa4e09099cef None admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:20.698926 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706bc6c-fff4-4e3b-a46c-fa4e09099cef None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:20.699208 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706bc6c-fff4-4e3b-a46c-fa4e09099cef None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:20.699852 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9706bc6c-fff4-4e3b-a46c-fa4e09099cef None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:20.701204 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9706bc6c-fff4-4e3b-a46c-fa4e09099cef None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:20.802948 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 517/1020] 10.222.0.82 () {60 vars in 1141 bytes} [Thu Aug 18 16:55:20 2022] POST /identity/v3/services => generated 242 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 16:55:20.816421 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5526a3-e713-47f5-b06c-6c7da2d20a64 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:20.816930 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5526a3-e713-47f5-b06c-6c7da2d20a64 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:20.816930 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5526a3-e713-47f5-b06c-6c7da2d20a64 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:20.818603 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 504/1021] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:55:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:55:20.824894 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d904ab-f572-4514-8859-07523190a392 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:20.825226 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d904ab-f572-4514-8859-07523190a392 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:20.825462 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d904ab-f572-4514-8859-07523190a392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:20.845453 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-82d904ab-f572-4514-8859-07523190a392 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:55:20.846798 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-82d904ab-f572-4514-8859-07523190a392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:55:20.849458 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:20.928952 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 518/1022] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:55:20 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 16:55:21.202612 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5526a3-e713-47f5-b06c-6c7da2d20a64 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:21.218530 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:21.307577 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5526a3-e713-47f5-b06c-6c7da2d20a64 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:21.342743 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7912b1e5-22f4-4166-b8c9-b17a8beea8e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:21.343410 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7912b1e5-22f4-4166-b8c9-b17a8beea8e8 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:21.343688 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7912b1e5-22f4-4166-b8c9-b17a8beea8e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:21.343972 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7912b1e5-22f4-4166-b8c9-b17a8beea8e8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:21.354264 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7912b1e5-22f4-4166-b8c9-b17a8beea8e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:55:21.358997 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-7912b1e5-22f4-4166-b8c9-b17a8beea8e8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 18 16:55:21.358997 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:21.358997 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:55:21.358997 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:55:21.358997 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:55:21.358997 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:55:21.358997 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:55:21.358997 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:55:21.358997 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:55:21.358997 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:55:21.358997 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:55:21.358997 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:55:21.358997 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:55:21.358997 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:55:21.358997 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:55:21.358997 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:55:21.359923 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:55:21.359923 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:55:21.359923 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:55:21.359923 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:55:21.359923 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:55:21.359923 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:55:21.359923 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:55:21.359923 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:55:21.359923 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:55:21.359923 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:55:21.359923 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:55:21.359923 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:55:21.359923 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:55:21.359923 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:55:21.359923 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:55:21.359923 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:55:21.360813 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:55:21.360813 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:55:21.360813 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:55:21.360813 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:55:21.360813 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:55:21.360813 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:55:21.360813 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:55:21.360813 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:55:21.360813 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:55:21.360813 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:55:21.360813 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 18 16:55:21.360813 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:55:21.360813 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 505/1023] 10.222.0.82 () {56 vars in 1120 bytes} [Thu Aug 18 16:55:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 16:55:21.368453 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d904ab-f572-4514-8859-07523190a392 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:21.468903 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d904ab-f572-4514-8859-07523190a392 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:21.503135 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cf8983-243d-4167-809b-450c23cce390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:21.503746 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf8983-243d-4167-809b-450c23cce390 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:21.504027 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf8983-243d-4167-809b-450c23cce390 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:21.504298 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf8983-243d-4167-809b-450c23cce390 None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:21.504948 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cf8983-243d-4167-809b-450c23cce390 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:21.505250 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cf8983-243d-4167-809b-450c23cce390 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:21.506607 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cf8983-243d-4167-809b-450c23cce390 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:21.508027 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:21.515729 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 519/1024] 10.222.0.82 () {56 vars in 1115 bytes} [Thu Aug 18 16:55:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:21.521455 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7912b1e5-22f4-4166-b8c9-b17a8beea8e8 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:21.598637 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:21.623611 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7912b1e5-22f4-4166-b8c9-b17a8beea8e8 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:21.658267 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ac97b9-793f-4d51-8ddc-2014cc93c17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:21.658931 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ac97b9-793f-4d51-8ddc-2014cc93c17c None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:21.659210 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ac97b9-793f-4d51-8ddc-2014cc93c17c None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:21.659484 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ac97b9-793f-4d51-8ddc-2014cc93c17c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:21.669778 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ac97b9-793f-4d51-8ddc-2014cc93c17c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:21.671264 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ac97b9-793f-4d51-8ddc-2014cc93c17c None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:21.685625 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-d6ac97b9-793f-4d51-8ddc-2014cc93c17c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:55:21.685625 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:21.685625 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:55:21.685625 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:55:21.685625 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:55:21.685625 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:55:21.685625 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:55:21.685625 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:55:21.685625 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:55:21.685625 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:55:21.685625 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:55:21.685625 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:55:21.685625 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:55:21.685625 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:55:21.685625 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:55:21.685625 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:55:21.686557 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:55:21.686557 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:55:21.686557 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:55:21.686557 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:55:21.686557 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:55:21.686557 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:55:21.686557 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:55:21.686557 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:55:21.686557 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:55:21.686557 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:55:21.686557 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:55:21.686557 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:55:21.686557 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:55:21.686557 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:55:21.686557 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:55:21.686557 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:55:21.686557 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:55:21.687848 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:55:21.687848 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:55:21.687848 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:55:21.691667 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 506/1025] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:55:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:55:21.693025 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cf8983-243d-4167-809b-450c23cce390 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:21.695256 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1b5785-72ad-4c34-b959-7c7a8bf2f912 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:21.695547 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1b5785-72ad-4c34-b959-7c7a8bf2f912 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:21.696372 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1b5785-72ad-4c34-b959-7c7a8bf2f912 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:21.697860 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 507/1026] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:55:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 16:55:21.702601 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b336e98-da0a-4a10-a972-bec83033f265 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:21.702892 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b336e98-da0a-4a10-a972-bec83033f265 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:21.703201 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b336e98-da0a-4a10-a972-bec83033f265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:21.721689 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-5b336e98-da0a-4a10-a972-bec83033f265 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:55:21.722803 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-5b336e98-da0a-4a10-a972-bec83033f265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:55:21.798926 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 508/1027] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:55:21 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 16:55:21.810490 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cf8983-243d-4167-809b-450c23cce390 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:21.845889 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64886e2-69e5-4a0d-b199-9975a127928c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:21.846626 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64886e2-69e5-4a0d-b199-9975a127928c None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:21.846932 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64886e2-69e5-4a0d-b199-9975a127928c None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:21.847207 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64886e2-69e5-4a0d-b199-9975a127928c None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:21.847846 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64886e2-69e5-4a0d-b199-9975a127928c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:21.848150 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64886e2-69e5-4a0d-b199-9975a127928c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:21.849509 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64886e2-69e5-4a0d-b199-9975a127928c None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:21.861664 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 520/1028] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:55:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:21.867323 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b336e98-da0a-4a10-a972-bec83033f265 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:21.927045 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:21.971122 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b336e98-da0a-4a10-a972-bec83033f265 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:22.005286 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03609dba-4c28-4a8a-a36f-c64f97f3dac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:22.005844 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03609dba-4c28-4a8a-a36f-c64f97f3dac1 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:22.006126 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03609dba-4c28-4a8a-a36f-c64f97f3dac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:22.006397 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03609dba-4c28-4a8a-a36f-c64f97f3dac1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:22.013249 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03609dba-4c28-4a8a-a36f-c64f97f3dac1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:22.014704 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03609dba-4c28-4a8a-a36f-c64f97f3dac1 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:22.026443 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-03609dba-4c28-4a8a-a36f-c64f97f3dac1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 18 16:55:22.026443 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:22.026443 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:55:22.026443 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:55:22.026443 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:55:22.026443 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:55:22.026443 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:55:22.026443 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:55:22.026443 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:55:22.026443 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:55:22.026443 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:55:22.026443 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:55:22.026443 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 16:55:22.026443 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 16:55:22.026443 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 16:55:22.026443 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 16:55:22.027599 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:55:22.027599 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:55:22.027599 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 16:55:22.027599 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:55:22.027599 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 16:55:22.027599 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:55:22.027599 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:55:22.027599 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:55:22.027599 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:55:22.027599 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:55:22.027599 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:55:22.027599 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:55:22.027599 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:55:22.027599 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:55:22.027599 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:55:22.027599 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:55:22.027599 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:55:22.027599 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:55:22.028555 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:55:22.028555 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:55:22.028555 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 16:55:22.028555 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 16:55:22.028555 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 16:55:22.028555 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 16:55:22.028555 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 16:55:22.028555 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 16:55:22.028555 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 18 16:55:22.028555 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:55:22.028555 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 509/1029] 10.222.0.82 () {56 vars in 1093 bytes} [Thu Aug 18 16:55:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 16:55:22.035269 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64886e2-69e5-4a0d-b199-9975a127928c None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:22.068267 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03609dba-4c28-4a8a-a36f-c64f97f3dac1 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:22.096939 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:22.142879 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64886e2-69e5-4a0d-b199-9975a127928c None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:22.180987 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f671620-92fc-4f14-864e-01957e00b33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:22.181608 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f671620-92fc-4f14-864e-01957e00b33d None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:22.181893 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f671620-92fc-4f14-864e-01957e00b33d None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:22.182219 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f671620-92fc-4f14-864e-01957e00b33d None admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:22.182987 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f671620-92fc-4f14-864e-01957e00b33d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:22.183296 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f671620-92fc-4f14-864e-01957e00b33d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:22.184726 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f671620-92fc-4f14-864e-01957e00b33d None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:22.185677 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03609dba-4c28-4a8a-a36f-c64f97f3dac1 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:22.202284 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:22.208271 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 521/1030] 10.222.0.82 () {56 vars in 1133 bytes} [Thu Aug 18 16:55:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:22.209958 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f671620-92fc-4f14-864e-01957e00b33d None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:22.221410 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172864bd-b916-4934-861c-451cd9113c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:22.222226 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172864bd-b916-4934-861c-451cd9113c82 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:22.223036 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172864bd-b916-4934-861c-451cd9113c82 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:22.223036 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172864bd-b916-4934-861c-451cd9113c82 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:22.223978 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172864bd-b916-4934-861c-451cd9113c82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:22.225494 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172864bd-b916-4934-861c-451cd9113c82 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:22.227748 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 510/1031] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:55:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:55:22.233658 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172864bd-b916-4934-861c-451cd9113c82 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:22.319051 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f671620-92fc-4f14-864e-01957e00b33d None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:22.335841 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:22.347238 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172864bd-b916-4934-861c-451cd9113c82 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:22.362937 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9196c356-fcca-4cb1-98b4-b9416d1409be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:22.363554 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9196c356-fcca-4cb1-98b4-b9416d1409be None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:22.363838 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9196c356-fcca-4cb1-98b4-b9416d1409be None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:22.364347 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9196c356-fcca-4cb1-98b4-b9416d1409be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:22.383014 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3498dc12-fd6e-4182-8124-edf1b1b8f20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:22.384230 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3498dc12-fd6e-4182-8124-edf1b1b8f20d None admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:22.384509 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3498dc12-fd6e-4182-8124-edf1b1b8f20d None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:22.384776 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3498dc12-fd6e-4182-8124-edf1b1b8f20d None admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:22.385405 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3498dc12-fd6e-4182-8124-edf1b1b8f20d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:22.386841 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3498dc12-fd6e-4182-8124-edf1b1b8f20d None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:22.436711 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9196c356-fcca-4cb1-98b4-b9416d1409be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:22.438153 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9196c356-fcca-4cb1-98b4-b9416d1409be None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:22.465219 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 511/1032] 10.222.0.82 () {60 vars in 1132 bytes} [Thu Aug 18 16:55:22 2022] POST /identity/v3/users => generated 285 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 16:55:22.554741 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 522/1033] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:55:22 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 18 16:55:22.557829 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3498dc12-fd6e-4182-8124-edf1b1b8f20d None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:22.604740 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:22.667087 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3498dc12-fd6e-4182-8124-edf1b1b8f20d None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:22.702175 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2ec1cf-df9b-43f7-9a75-9d705ed49c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:22.702821 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2ec1cf-df9b-43f7-9a75-9d705ed49c18 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:22.703143 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2ec1cf-df9b-43f7-9a75-9d705ed49c18 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:22.703421 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2ec1cf-df9b-43f7-9a75-9d705ed49c18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:22.715340 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2ec1cf-df9b-43f7-9a75-9d705ed49c18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:22.716803 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2ec1cf-df9b-43f7-9a75-9d705ed49c18 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:22.732019 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-7e2ec1cf-df9b-43f7-9a75-9d705ed49c18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:55:22.732019 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:22.732019 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:55:22.732019 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:55:22.732019 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:55:22.732019 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:55:22.732019 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:55:22.732019 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:55:22.732019 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:55:22.732019 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:55:22.732019 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:55:22.732019 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:55:22.732019 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:55:22.732019 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:55:22.732019 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:55:22.732019 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:55:22.732965 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:55:22.732965 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:55:22.732965 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:55:22.732965 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:55:22.732965 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:55:22.732965 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:55:22.732965 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:55:22.732965 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:55:22.732965 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:55:22.732965 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:55:22.732965 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:55:22.732965 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:55:22.732965 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:55:22.732965 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:55:22.732965 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:55:22.732965 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:55:22.732965 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:55:22.738461 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:55:22.738461 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:55:22.738461 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:55:22.738461 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 512/1034] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:55:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 16:55:22.739836 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9196c356-fcca-4cb1-98b4-b9416d1409be None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:22.750816 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e58c764-24dc-4d71-b1b5-421e898a946d None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:22.751149 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e58c764-24dc-4d71-b1b5-421e898a946d None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:22.751431 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e58c764-24dc-4d71-b1b5-421e898a946d None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:22.752813 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 513/1035] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:55:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 16:55:22.755952 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:22.757727 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e26a5a1-a3c1-4c4e-be50-0c941e89b354 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:22.758015 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e26a5a1-a3c1-4c4e-be50-0c941e89b354 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:22.758324 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e26a5a1-a3c1-4c4e-be50-0c941e89b354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:22.777068 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-3e26a5a1-a3c1-4c4e-be50-0c941e89b354 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:55:22.778268 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-3e26a5a1-a3c1-4c4e-be50-0c941e89b354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:55:22.854135 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 514/1036] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:55:22 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 16:55:22.855215 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9196c356-fcca-4cb1-98b4-b9416d1409be None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:22.890260 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09712944-a0b2-4321-af2b-8888c07caac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:22.890866 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09712944-a0b2-4321-af2b-8888c07caac0 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:22.891176 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09712944-a0b2-4321-af2b-8888c07caac0 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:22.891450 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09712944-a0b2-4321-af2b-8888c07caac0 None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:22.892084 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09712944-a0b2-4321-af2b-8888c07caac0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:22.892385 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09712944-a0b2-4321-af2b-8888c07caac0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:22.893736 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09712944-a0b2-4321-af2b-8888c07caac0 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:22.910079 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 523/1037] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:55:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:22.913721 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e26a5a1-a3c1-4c4e-be50-0c941e89b354 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:22.936830 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:23.015400 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e26a5a1-a3c1-4c4e-be50-0c941e89b354 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:23.049729 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd090f3-a1f2-4326-85ae-2c8e43fc203c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:23.050358 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd090f3-a1f2-4326-85ae-2c8e43fc203c None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:23.050636 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd090f3-a1f2-4326-85ae-2c8e43fc203c None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:23.050937 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd090f3-a1f2-4326-85ae-2c8e43fc203c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:23.057613 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd090f3-a1f2-4326-85ae-2c8e43fc203c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:23.059080 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd090f3-a1f2-4326-85ae-2c8e43fc203c None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:23.069922 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-5fd090f3-a1f2-4326-85ae-2c8e43fc203c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:55:23.069922 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:23.069922 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:55:23.069922 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:55:23.069922 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:55:23.069922 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:55:23.069922 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:55:23.069922 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:55:23.069922 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:55:23.069922 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:55:23.069922 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:55:23.069922 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:55:23.069922 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 16:55:23.069922 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 16:55:23.069922 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 16:55:23.072001 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 16:55:23.072001 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:55:23.072001 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:55:23.072001 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:55:23.072001 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:55:23.072001 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:55:23.072001 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:55:23.072001 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:55:23.072001 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:55:23.072001 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:55:23.072001 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:55:23.072001 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:55:23.072001 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:55:23.072001 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:55:23.072001 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:55:23.072001 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:55:23.072001 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:55:23.072904 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 16:55:23.072904 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 16:55:23.072904 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 16:55:23.072904 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 16:55:23.072904 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 16:55:23.072904 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 16:55:23.072904 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:55:23.072904 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:55:23.072904 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 515/1038] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 16:55:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:55:23.077319 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09712944-a0b2-4321-af2b-8888c07caac0 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:23.132330 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd090f3-a1f2-4326-85ae-2c8e43fc203c None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:23.154251 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:23.181679 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09712944-a0b2-4321-af2b-8888c07caac0 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:23.217757 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de39a18-099d-46a3-afd9-bfba285ee3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:23.218370 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de39a18-099d-46a3-afd9-bfba285ee3ae None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:23.218652 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de39a18-099d-46a3-afd9-bfba285ee3ae None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:23.218951 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de39a18-099d-46a3-afd9-bfba285ee3ae None admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:23.219638 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de39a18-099d-46a3-afd9-bfba285ee3ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:23.219946 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de39a18-099d-46a3-afd9-bfba285ee3ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:23.221369 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de39a18-099d-46a3-afd9-bfba285ee3ae None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:23.222209 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:23.233540 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 524/1039] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 16:55:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:55:23.239465 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de39a18-099d-46a3-afd9-bfba285ee3ae None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:23.247054 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd090f3-a1f2-4326-85ae-2c8e43fc203c None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:23.285796 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bedb4c4-cec4-41f9-a248-02e8a778a6d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:23.286430 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bedb4c4-cec4-41f9-a248-02e8a778a6d2 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:23.286716 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bedb4c4-cec4-41f9-a248-02e8a778a6d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:23.287028 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bedb4c4-cec4-41f9-a248-02e8a778a6d2 None admin] PATH_INFO: `/v3/services/image` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:23.287610 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bedb4c4-cec4-41f9-a248-02e8a778a6d2 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:23.289085 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bedb4c4-cec4-41f9-a248-02e8a778a6d2 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:23.297928 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-1bedb4c4-cec4-41f9-a248-02e8a778a6d2 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 18 16:55:23.297928 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:23.297928 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 16:55:23.297928 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 16:55:23.297928 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 16:55:23.297928 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 16:55:23.297928 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 16:55:23.297928 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 16:55:23.297928 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 18 16:55:23.297928 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:55:23.297928 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 16:55:23.297928 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:55:23.297928 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:23.297928 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:55:23.297928 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:55:23.297928 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:55:23.297928 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:55:23.298952 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:55:23.298952 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:55:23.298952 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:55:23.298952 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:55:23.298952 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:55:23.298952 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:55:23.298952 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 16:55:23.298952 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 16:55:23.298952 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 16:55:23.298952 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 16:55:23.298952 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:55:23.298952 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:55:23.298952 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:55:23.298952 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:55:23.298952 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:55:23.298952 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:55:23.299804 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:55:23.299804 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:55:23.299804 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:55:23.299804 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:55:23.299804 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:55:23.299804 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:55:23.299804 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:55:23.299804 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:55:23.299804 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:55:23.299804 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:55:23.299804 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 16:55:23.299804 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 16:55:23.299804 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 18 16:55:23.299804 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:55:23.303977 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 516/1040] 10.222.0.82 () {56 vars in 1105 bytes} [Thu Aug 18 16:55:23 2022] GET /identity/v3/services/image => generated 86 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 16:55:23.304711 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bedb4c4-cec4-41f9-a248-02e8a778a6d2 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:23.350070 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de39a18-099d-46a3-afd9-bfba285ee3ae None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:23.386530 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5555b254-08e6-4b10-9bac-6e7bb4c8f444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:23.387089 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5555b254-08e6-4b10-9bac-6e7bb4c8f444 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:23.387418 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5555b254-08e6-4b10-9bac-6e7bb4c8f444 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:23.387690 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5555b254-08e6-4b10-9bac-6e7bb4c8f444 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:23.388400 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5555b254-08e6-4b10-9bac-6e7bb4c8f444 None admin] RBAC: Adding query filter params (role.id=bae4160aa9de4d69b42641fd18c53e55, scope.project.id=2293c0aab61f44849d0d48b214cde7ea, user.id=55723c756cae4e7ab636054db48de02d) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:23.388767 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5555b254-08e6-4b10-9bac-6e7bb4c8f444 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:23.390199 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5555b254-08e6-4b10-9bac-6e7bb4c8f444 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:23.392004 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:23.406935 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 525/1041] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 16:55:23 2022] GET /identity/v3/role_assignments?user.id=55723c756cae4e7ab636054db48de02d&scope.project.id=2293c0aab61f44849d0d48b214cde7ea&role.id=bae4160aa9de4d69b42641fd18c53e55 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:55:23.422317 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bedb4c4-cec4-41f9-a248-02e8a778a6d2 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:23.463902 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de7c5c3-4551-4b1f-ad7d-19ce89db1feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:23.464535 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de7c5c3-4551-4b1f-ad7d-19ce89db1feb None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:23.464823 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de7c5c3-4551-4b1f-ad7d-19ce89db1feb None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:23.465097 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de7c5c3-4551-4b1f-ad7d-19ce89db1feb None admin] PATH_INFO: `/v3/services` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:23.465739 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de7c5c3-4551-4b1f-ad7d-19ce89db1feb None admin] RBAC: Adding query filter params (name=image) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:23.466045 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de7c5c3-4551-4b1f-ad7d-19ce89db1feb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:23.467589 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de7c5c3-4551-4b1f-ad7d-19ce89db1feb None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:23.478606 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 517/1042] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:55:23 2022] GET /identity/v3/services?name=image => generated 122 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:55:23.480321 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5555b254-08e6-4b10-9bac-6e7bb4c8f444 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:23.589729 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5555b254-08e6-4b10-9bac-6e7bb4c8f444 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:23.626982 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f22a2cd-3be0-420c-9782-1cc035b3a5f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:23.627646 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f22a2cd-3be0-420c-9782-1cc035b3a5f6 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:23.627930 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f22a2cd-3be0-420c-9782-1cc035b3a5f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:23.628203 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f22a2cd-3be0-420c-9782-1cc035b3a5f6 None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:23.628829 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f22a2cd-3be0-420c-9782-1cc035b3a5f6 None admin] RBAC: Adding query filter params (type=image) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:23.629133 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f22a2cd-3be0-420c-9782-1cc035b3a5f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:23.630513 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f22a2cd-3be0-420c-9782-1cc035b3a5f6 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:23.640346 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 526/1043] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:55:23 2022] GET /identity/v3/services?type=image => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:23.641590 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de7c5c3-4551-4b1f-ad7d-19ce89db1feb None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:23.654503 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:23.744037 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de7c5c3-4551-4b1f-ad7d-19ce89db1feb None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:23.783838 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38de427-2bdc-469e-afb5-e5e34d6eca2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:23.784454 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38de427-2bdc-469e-afb5-e5e34d6eca2a None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:23.784740 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38de427-2bdc-469e-afb5-e5e34d6eca2a None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:23.785017 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38de427-2bdc-469e-afb5-e5e34d6eca2a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:23.785720 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38de427-2bdc-469e-afb5-e5e34d6eca2a None admin] RBAC: Adding query filter params (interface=public, service_id=3e0b0df034214147866f6882a14e6e4d, region_id=RegionOne) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:23.786032 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38de427-2bdc-469e-afb5-e5e34d6eca2a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:23.787454 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38de427-2bdc-469e-afb5-e5e34d6eca2a None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:23.797907 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 518/1044] 10.222.0.82 () {56 vars in 1251 bytes} [Thu Aug 18 16:55:23 2022] GET /identity/v3/endpoints?service_id=3e0b0df034214147866f6882a14e6e4d&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:23.799443 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f22a2cd-3be0-420c-9782-1cc035b3a5f6 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:23.864941 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:23.905693 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f22a2cd-3be0-420c-9782-1cc035b3a5f6 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:23.939447 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b27aa19-b261-45b0-8872-86e3595b9dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:23.940240 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b27aa19-b261-45b0-8872-86e3595b9dd8 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:23.940623 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b27aa19-b261-45b0-8872-86e3595b9dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:23.941002 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b27aa19-b261-45b0-8872-86e3595b9dd8 None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:23.941715 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b27aa19-b261-45b0-8872-86e3595b9dd8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:23.943213 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b27aa19-b261-45b0-8872-86e3595b9dd8 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:23.958934 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 527/1045] 10.222.0.82 () {56 vars in 1087 bytes} [Thu Aug 18 16:55:23 2022] GET /identity/v3/services => generated 1957 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 16:55:24.442487 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f79fad-e5b2-4bf2-9313-de752301a43b None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:24.443041 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f79fad-e5b2-4bf2-9313-de752301a43b None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:24.443041 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f79fad-e5b2-4bf2-9313-de752301a43b None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:24.444536 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 519/1046] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:55:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:55:24.450436 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce62e1e-35d9-4e4e-a248-bdad907ebf2c None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:24.450725 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce62e1e-35d9-4e4e-a248-bdad907ebf2c None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:24.451031 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce62e1e-35d9-4e4e-a248-bdad907ebf2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:24.470816 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-3ce62e1e-35d9-4e4e-a248-bdad907ebf2c None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:55:24.472028 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-3ce62e1e-35d9-4e4e-a248-bdad907ebf2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:55:24.543707 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 528/1047] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:55:24 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 16:55:24.805820 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f79fad-e5b2-4bf2-9313-de752301a43b None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:24.885835 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:24.911723 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f79fad-e5b2-4bf2-9313-de752301a43b None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:24.948572 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695642e8-1eaa-4a6a-b41f-f56e9bd3ae2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:24.949510 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695642e8-1eaa-4a6a-b41f-f56e9bd3ae2a None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:24.949978 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695642e8-1eaa-4a6a-b41f-f56e9bd3ae2a None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:24.950469 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695642e8-1eaa-4a6a-b41f-f56e9bd3ae2a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:24.961119 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695642e8-1eaa-4a6a-b41f-f56e9bd3ae2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:55:24.966535 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-695642e8-1eaa-4a6a-b41f-f56e9bd3ae2a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 16:55:24.966535 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:24.966535 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:55:24.966535 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:55:24.966535 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:55:24.966535 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:55:24.966535 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:55:24.966535 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:55:24.966535 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:55:24.966535 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:55:24.966535 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:55:24.966535 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:55:24.966535 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:55:24.966535 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:55:24.966535 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:55:24.966535 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:55:24.967534 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:55:24.967534 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:55:24.967534 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:55:24.967534 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:55:24.967534 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:55:24.967534 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:55:24.967534 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:55:24.967534 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:55:24.967534 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:55:24.967534 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:55:24.967534 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:55:24.967534 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:55:24.967534 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:55:24.967534 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:55:24.967534 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:55:24.967534 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:55:24.968363 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:55:24.968363 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:55:24.968363 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:55:24.968363 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:55:24.968363 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:55:24.968363 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:55:24.968363 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:55:24.968363 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:55:24.968363 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:55:24.968363 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:55:24.968363 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 16:55:24.968363 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:55:24.969040 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 520/1048] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 16:55:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:55:24.974515 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce62e1e-35d9-4e4e-a248-bdad907ebf2c None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:25.033592 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:25.080238 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce62e1e-35d9-4e4e-a248-bdad907ebf2c None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:25.114076 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d40a08-d0ae-41b9-985a-2c501b4e539f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:25.114678 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d40a08-d0ae-41b9-985a-2c501b4e539f None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:25.115141 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d40a08-d0ae-41b9-985a-2c501b4e539f None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:25.115420 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d40a08-d0ae-41b9-985a-2c501b4e539f None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:25.116066 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d40a08-d0ae-41b9-985a-2c501b4e539f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:25.116361 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d40a08-d0ae-41b9-985a-2c501b4e539f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:25.117736 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d40a08-d0ae-41b9-985a-2c501b4e539f None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:25.130522 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 529/1049] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:55:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:55:25.131759 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695642e8-1eaa-4a6a-b41f-f56e9bd3ae2a None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:25.231392 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695642e8-1eaa-4a6a-b41f-f56e9bd3ae2a None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:25.264507 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27ed304-188f-4e0d-a042-fd7ec2b7f9f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:25.265133 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27ed304-188f-4e0d-a042-fd7ec2b7f9f6 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:25.265411 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27ed304-188f-4e0d-a042-fd7ec2b7f9f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:25.265681 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27ed304-188f-4e0d-a042-fd7ec2b7f9f6 None admin] PATH_INFO: `/v3/users/d93b0f69c8314968964278e6a4d18d34` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:25.267586 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:25.280117 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27ed304-188f-4e0d-a042-fd7ec2b7f9f6 None admin] RBAC: Authorizing `identity:get_user(user_id=d93b0f69c8314968964278e6a4d18d34)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:25.281556 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27ed304-188f-4e0d-a042-fd7ec2b7f9f6 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:25.283416 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 521/1050] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 16:55:25 2022] GET /identity/v3/users/d93b0f69c8314968964278e6a4d18d34 => generated 285 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:55:25.289404 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d40a08-d0ae-41b9-985a-2c501b4e539f None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:25.293248 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02d287-2c24-40bb-b5b8-c1036ae52dfa None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:25.293542 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02d287-2c24-40bb-b5b8-c1036ae52dfa None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:25.293824 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02d287-2c24-40bb-b5b8-c1036ae52dfa None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:25.295157 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 522/1051] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:55:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 16:55:25.300160 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f01c02f-0893-4394-abe6-6247111bd22d None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:25.300450 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f01c02f-0893-4394-abe6-6247111bd22d None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:25.300730 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f01c02f-0893-4394-abe6-6247111bd22d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:25.318209 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-9f01c02f-0893-4394-abe6-6247111bd22d None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:55:25.319435 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-9f01c02f-0893-4394-abe6-6247111bd22d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:55:25.398509 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d40a08-d0ae-41b9-985a-2c501b4e539f None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:25.399678 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 523/1052] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:55:25 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 16:55:25.416306 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:25.433207 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8e378e-bae2-4a44-bc46-cd29399fb427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:25.433829 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e378e-bae2-4a44-bc46-cd29399fb427 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:25.434111 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e378e-bae2-4a44-bc46-cd29399fb427 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:25.434381 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e378e-bae2-4a44-bc46-cd29399fb427 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:25.502187 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8e378e-bae2-4a44-bc46-cd29399fb427 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:25.503663 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8e378e-bae2-4a44-bc46-cd29399fb427 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:25.606608 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 530/1053] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:55:25 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 16:55:25.608504 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f01c02f-0893-4394-abe6-6247111bd22d None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:25.659091 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8e378e-bae2-4a44-bc46-cd29399fb427 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:25.678021 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:25.713820 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f01c02f-0893-4394-abe6-6247111bd22d None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:25.738280 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:25.751026 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890fc079-6f81-4670-9c8e-a9c02c64b24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:25.751689 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890fc079-6f81-4670-9c8e-a9c02c64b24b None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:25.752051 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890fc079-6f81-4670-9c8e-a9c02c64b24b None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:25.752329 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890fc079-6f81-4670-9c8e-a9c02c64b24b None admin] PATH_INFO: `/v3/projects/b83fb425981746e3baea3443652d4a7a` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:25.753544 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890fc079-6f81-4670-9c8e-a9c02c64b24b None admin] RBAC: Authorizing `identity:get_project(project_id=b83fb425981746e3baea3443652d4a7a)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:25.755120 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890fc079-6f81-4670-9c8e-a9c02c64b24b None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:25.762292 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 524/1054] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 16:55:25 2022] GET /identity/v3/projects/b83fb425981746e3baea3443652d4a7a => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:55:25.764198 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890fc079-6f81-4670-9c8e-a9c02c64b24b None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:25.770494 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8e378e-bae2-4a44-bc46-cd29399fb427 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:25.806624 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0905a8-e0be-456b-95f6-b33d39ff33ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:25.807296 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0905a8-e0be-456b-95f6-b33d39ff33ad None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:25.807581 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0905a8-e0be-456b-95f6-b33d39ff33ad None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:25.807858 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0905a8-e0be-456b-95f6-b33d39ff33ad None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:25.818241 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0905a8-e0be-456b-95f6-b33d39ff33ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:55:25.823627 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-fd0905a8-e0be-456b-95f6-b33d39ff33ad None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 18 16:55:25.823627 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:25.823627 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:55:25.823627 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:55:25.823627 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:55:25.823627 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:55:25.823627 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:55:25.823627 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:55:25.823627 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:55:25.823627 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:55:25.823627 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:55:25.823627 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:55:25.823627 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:55:25.823627 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:55:25.823627 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:55:25.823627 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:55:25.824562 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:55:25.824562 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:55:25.824562 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:55:25.824562 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:55:25.824562 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:55:25.824562 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:55:25.824562 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:55:25.824562 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:55:25.824562 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:55:25.824562 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:55:25.824562 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:55:25.824562 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:55:25.824562 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:55:25.824562 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:55:25.824562 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:55:25.824562 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:55:25.825380 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:55:25.825380 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:55:25.825380 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:55:25.825380 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:55:25.825380 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:55:25.825380 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:55:25.825380 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:55:25.825380 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:55:25.825380 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:55:25.825380 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:55:25.825380 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 18 16:55:25.825380 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:55:25.825380 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 531/1055] 10.222.0.82 () {56 vars in 1120 bytes} [Thu Aug 18 16:55:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 16:55:25.830671 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0905a8-e0be-456b-95f6-b33d39ff33ad None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:25.880454 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890fc079-6f81-4670-9c8e-a9c02c64b24b None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:25.916286 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab395670-b322-4f39-a833-a74869af46ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:25.916895 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab395670-b322-4f39-a833-a74869af46ed None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:25.917177 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab395670-b322-4f39-a833-a74869af46ed None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:25.917451 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab395670-b322-4f39-a833-a74869af46ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:25.918172 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab395670-b322-4f39-a833-a74869af46ed None admin] RBAC: Adding query filter params (role.id=2c6966cfb10f4d72b0242e9fb5e13b17, scope.project.id=b83fb425981746e3baea3443652d4a7a, user.id=d93b0f69c8314968964278e6a4d18d34) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:25.918480 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab395670-b322-4f39-a833-a74869af46ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:25.919944 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab395670-b322-4f39-a833-a74869af46ed None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:25.923228 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:25.925594 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 525/1056] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 16:55:25 2022] GET /identity/v3/role_assignments?user.id=d93b0f69c8314968964278e6a4d18d34&scope.project.id=b83fb425981746e3baea3443652d4a7a&role.id=2c6966cfb10f4d72b0242e9fb5e13b17 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:25.926755 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3514ce-0127-497c-a821-f753dc907098 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:25.927064 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3514ce-0127-497c-a821-f753dc907098 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:25.927346 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3514ce-0127-497c-a821-f753dc907098 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:25.928730 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 526/1057] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:55:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:55:25.935044 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fda312-fd3c-49cf-9759-91a12b9ddca6 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:25.935331 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fda312-fd3c-49cf-9759-91a12b9ddca6 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:25.935606 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fda312-fd3c-49cf-9759-91a12b9ddca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:25.952233 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0905a8-e0be-456b-95f6-b33d39ff33ad None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:25.955236 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-09fda312-fd3c-49cf-9759-91a12b9ddca6 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:55:25.956465 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-09fda312-fd3c-49cf-9759-91a12b9ddca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:55:25.992026 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41082c7f-ce24-4744-9847-839ffed5bcf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:25.993191 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41082c7f-ce24-4744-9847-839ffed5bcf1 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:25.993191 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41082c7f-ce24-4744-9847-839ffed5bcf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:25.993191 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41082c7f-ce24-4744-9847-839ffed5bcf1 None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:25.994401 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41082c7f-ce24-4744-9847-839ffed5bcf1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:25.994708 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41082c7f-ce24-4744-9847-839ffed5bcf1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:25.996151 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41082c7f-ce24-4744-9847-839ffed5bcf1 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:26.006513 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 532/1058] 10.222.0.82 () {56 vars in 1115 bytes} [Thu Aug 18 16:55:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:55:26.012634 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41082c7f-ce24-4744-9847-839ffed5bcf1 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:26.015907 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:26.036492 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 527/1059] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:55:25 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 16:55:26.122203 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41082c7f-ce24-4744-9847-839ffed5bcf1 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:26.155505 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfa8ebf-2bf9-4acd-a282-f7bdffd71fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:26.156059 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfa8ebf-2bf9-4acd-a282-f7bdffd71fe9 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:26.156336 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfa8ebf-2bf9-4acd-a282-f7bdffd71fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:26.156604 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfa8ebf-2bf9-4acd-a282-f7bdffd71fe9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:26.165779 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:26.168529 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfa8ebf-2bf9-4acd-a282-f7bdffd71fe9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:26.169966 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfa8ebf-2bf9-4acd-a282-f7bdffd71fe9 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:26.184364 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-cbfa8ebf-2bf9-4acd-a282-f7bdffd71fe9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:55:26.184364 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:26.184364 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:55:26.184364 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:55:26.184364 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:55:26.184364 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:55:26.184364 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:55:26.184364 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:55:26.184364 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:55:26.184364 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:55:26.184364 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:55:26.184364 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:55:26.184364 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:55:26.184364 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:55:26.184364 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:55:26.184364 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:55:26.185302 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:55:26.185302 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:55:26.185302 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:55:26.185302 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:55:26.185302 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:55:26.185302 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:55:26.185302 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:55:26.185302 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:55:26.185302 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:55:26.185302 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:55:26.185302 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:55:26.185302 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:55:26.185302 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:55:26.185302 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:55:26.185302 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:55:26.185302 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:55:26.185302 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:55:26.186199 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:55:26.186199 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:55:26.186199 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:55:26.186199 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 533/1060] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:55:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 16:55:26.191304 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fda312-fd3c-49cf-9759-91a12b9ddca6 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:26.294665 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fda312-fd3c-49cf-9759-91a12b9ddca6 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:26.312211 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfa8ebf-2bf9-4acd-a282-f7bdffd71fe9 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:26.329129 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903399e3-63bb-43b7-a13a-d348a3aa8a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:26.329744 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903399e3-63bb-43b7-a13a-d348a3aa8a6c None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:26.330033 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903399e3-63bb-43b7-a13a-d348a3aa8a6c None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:26.330310 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903399e3-63bb-43b7-a13a-d348a3aa8a6c None admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:26.330983 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903399e3-63bb-43b7-a13a-d348a3aa8a6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:26.331320 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903399e3-63bb-43b7-a13a-d348a3aa8a6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:26.332676 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903399e3-63bb-43b7-a13a-d348a3aa8a6c None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:26.349382 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 528/1061] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:55:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:26.350158 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903399e3-63bb-43b7-a13a-d348a3aa8a6c None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:26.403250 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:26.420434 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfa8ebf-2bf9-4acd-a282-f7bdffd71fe9 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:26.459913 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903399e3-63bb-43b7-a13a-d348a3aa8a6c None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:26.460738 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5288636-97e5-4793-9f6d-8dea2e131484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:26.462703 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5288636-97e5-4793-9f6d-8dea2e131484 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:26.463983 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5288636-97e5-4793-9f6d-8dea2e131484 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:26.464444 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5288636-97e5-4793-9f6d-8dea2e131484 None admin] PATH_INFO: `/v3/services/image` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:26.465203 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5288636-97e5-4793-9f6d-8dea2e131484 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:26.466803 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5288636-97e5-4793-9f6d-8dea2e131484 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:26.477518 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-a5288636-97e5-4793-9f6d-8dea2e131484 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 18 16:55:26.477518 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:26.477518 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 16:55:26.477518 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 16:55:26.477518 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 16:55:26.477518 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 16:55:26.477518 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 16:55:26.477518 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 16:55:26.477518 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 18 16:55:26.477518 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:55:26.477518 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 16:55:26.477518 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:55:26.477518 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:26.477518 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:55:26.477518 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:55:26.477518 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:55:26.477518 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:55:26.478526 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:55:26.478526 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:55:26.478526 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:55:26.478526 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:55:26.478526 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:55:26.478526 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:55:26.478526 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 16:55:26.478526 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 16:55:26.478526 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 16:55:26.478526 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 16:55:26.478526 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:55:26.478526 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:55:26.478526 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:55:26.478526 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:55:26.478526 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:55:26.478526 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:55:26.479416 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:55:26.479416 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:55:26.479416 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:55:26.479416 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:55:26.479416 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:55:26.479416 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:55:26.479416 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:55:26.479416 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:55:26.479416 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:55:26.479416 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:55:26.479416 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 16:55:26.479416 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 16:55:26.479416 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 18 16:55:26.479416 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:55:26.479416 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 534/1062] 10.222.0.82 () {56 vars in 1105 bytes} [Thu Aug 18 16:55:26 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 16:55:26.484492 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5288636-97e5-4793-9f6d-8dea2e131484 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:26.500405 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e3321e-08b4-48a6-86e4-944bbae6fe22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:26.501041 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3321e-08b4-48a6-86e4-944bbae6fe22 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:26.501325 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3321e-08b4-48a6-86e4-944bbae6fe22 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:26.501599 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3321e-08b4-48a6-86e4-944bbae6fe22 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:26.510209 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e3321e-08b4-48a6-86e4-944bbae6fe22 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:26.511715 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e3321e-08b4-48a6-86e4-944bbae6fe22 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:26.524614 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-e0e3321e-08b4-48a6-86e4-944bbae6fe22 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 18 16:55:26.524614 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:26.524614 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:55:26.524614 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:55:26.524614 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:55:26.524614 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:55:26.524614 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:55:26.524614 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:55:26.524614 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:55:26.524614 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:55:26.524614 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:55:26.524614 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:55:26.524614 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 16:55:26.524614 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 16:55:26.524614 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 16:55:26.524614 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 16:55:26.525534 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:55:26.525534 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:55:26.525534 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 16:55:26.525534 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:55:26.525534 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 16:55:26.525534 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:55:26.525534 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:55:26.525534 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:55:26.525534 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:55:26.525534 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:55:26.525534 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:55:26.525534 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:55:26.525534 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:55:26.525534 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:55:26.525534 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:55:26.525534 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:55:26.525534 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:55:26.525534 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:55:26.526469 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:55:26.526469 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:55:26.526469 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 16:55:26.526469 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 16:55:26.526469 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 16:55:26.526469 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 16:55:26.526469 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 16:55:26.526469 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 16:55:26.526469 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 18 16:55:26.526469 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:55:26.526469 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 529/1063] 10.222.0.82 () {56 vars in 1093 bytes} [Thu Aug 18 16:55:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:55:26.531323 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e3321e-08b4-48a6-86e4-944bbae6fe22 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:26.554018 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:26.601148 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5288636-97e5-4793-9f6d-8dea2e131484 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:26.635401 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75592fb2-9579-462d-8fbe-b59e71db3762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:26.636016 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75592fb2-9579-462d-8fbe-b59e71db3762 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:26.636279 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75592fb2-9579-462d-8fbe-b59e71db3762 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:26.636551 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75592fb2-9579-462d-8fbe-b59e71db3762 None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:26.637172 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75592fb2-9579-462d-8fbe-b59e71db3762 None admin] RBAC: Adding query filter params (name=image) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:26.637475 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75592fb2-9579-462d-8fbe-b59e71db3762 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:26.638830 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75592fb2-9579-462d-8fbe-b59e71db3762 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:26.644070 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 535/1064] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:55:26 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:55:26.645217 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e3321e-08b4-48a6-86e4-944bbae6fe22 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:26.649667 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75592fb2-9579-462d-8fbe-b59e71db3762 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:26.650107 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:26.683594 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a561db0-ef55-45a9-a379-c530b2458e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:26.684690 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a561db0-ef55-45a9-a379-c530b2458e25 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:26.685427 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a561db0-ef55-45a9-a379-c530b2458e25 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:26.685698 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a561db0-ef55-45a9-a379-c530b2458e25 None admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:26.686478 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a561db0-ef55-45a9-a379-c530b2458e25 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:26.686784 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a561db0-ef55-45a9-a379-c530b2458e25 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:26.688295 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a561db0-ef55-45a9-a379-c530b2458e25 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:26.712704 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 530/1065] 10.222.0.82 () {56 vars in 1133 bytes} [Thu Aug 18 16:55:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:55:26.713941 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a561db0-ef55-45a9-a379-c530b2458e25 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:26.756149 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75592fb2-9579-462d-8fbe-b59e71db3762 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:26.779044 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:26.797933 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff918a8-9dbf-42b8-bbad-904c2c6a423d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:26.798663 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff918a8-9dbf-42b8-bbad-904c2c6a423d None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:26.799058 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff918a8-9dbf-42b8-bbad-904c2c6a423d None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:26.799408 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff918a8-9dbf-42b8-bbad-904c2c6a423d None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:26.800109 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff918a8-9dbf-42b8-bbad-904c2c6a423d None admin] RBAC: Adding query filter params (type=image) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:26.800480 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff918a8-9dbf-42b8-bbad-904c2c6a423d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:26.801953 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff918a8-9dbf-42b8-bbad-904c2c6a423d None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:26.808451 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 536/1066] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:55:26 2022] GET /identity/v3/services?type=image => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:55:26.813882 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff918a8-9dbf-42b8-bbad-904c2c6a423d None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:26.827989 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a561db0-ef55-45a9-a379-c530b2458e25 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:26.864926 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16031c1-dbc1-4ab5-8186-39f88eb14a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:26.865564 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16031c1-dbc1-4ab5-8186-39f88eb14a73 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:26.865874 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16031c1-dbc1-4ab5-8186-39f88eb14a73 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:26.866149 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16031c1-dbc1-4ab5-8186-39f88eb14a73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:26.878541 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16031c1-dbc1-4ab5-8186-39f88eb14a73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:26.880242 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16031c1-dbc1-4ab5-8186-39f88eb14a73 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:26.896325 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-b16031c1-dbc1-4ab5-8186-39f88eb14a73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:55:26.896325 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:26.896325 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:55:26.896325 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:55:26.896325 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:55:26.896325 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:55:26.896325 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:55:26.896325 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:55:26.896325 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:55:26.896325 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:55:26.896325 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:55:26.896325 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:55:26.896325 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:55:26.896325 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:55:26.896325 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:55:26.896325 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:55:26.897349 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:55:26.897349 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:55:26.897349 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:55:26.897349 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:55:26.897349 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:55:26.897349 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:55:26.897349 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:55:26.897349 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:55:26.897349 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:55:26.897349 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:55:26.897349 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:55:26.897349 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:55:26.897349 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:55:26.897349 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:55:26.897349 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:55:26.897349 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:55:26.897349 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:55:26.898334 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:55:26.898334 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:55:26.898334 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:55:26.903057 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 531/1067] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:55:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 16:55:26.904480 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16031c1-dbc1-4ab5-8186-39f88eb14a73 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:26.920748 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:26.927868 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff918a8-9dbf-42b8-bbad-904c2c6a423d None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:26.964181 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e351e2-d520-42b9-8cbf-800b35eb1b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:26.964705 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e351e2-d520-42b9-8cbf-800b35eb1b03 None admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:26.964983 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e351e2-d520-42b9-8cbf-800b35eb1b03 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:26.965255 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e351e2-d520-42b9-8cbf-800b35eb1b03 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:26.965898 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e351e2-d520-42b9-8cbf-800b35eb1b03 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:26.967352 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e351e2-d520-42b9-8cbf-800b35eb1b03 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:27.009019 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16031c1-dbc1-4ab5-8186-39f88eb14a73 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:27.045069 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 537/1068] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 16:55:26 2022] POST /identity/v3/endpoints => generated 327 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 16:55:27.051495 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd663547-e35d-4272-a621-d9966d6fc291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:27.052129 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd663547-e35d-4272-a621-d9966d6fc291 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:27.052409 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd663547-e35d-4272-a621-d9966d6fc291 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:27.052687 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd663547-e35d-4272-a621-d9966d6fc291 None admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:27.053351 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd663547-e35d-4272-a621-d9966d6fc291 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:27.053656 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd663547-e35d-4272-a621-d9966d6fc291 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:27.055311 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd663547-e35d-4272-a621-d9966d6fc291 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:27.074527 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 532/1069] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:55:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:27.076274 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e351e2-d520-42b9-8cbf-800b35eb1b03 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:27.123249 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:27.186848 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e351e2-d520-42b9-8cbf-800b35eb1b03 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:27.224568 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5df711-4d82-4ea4-be6f-8218eb42cf40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:27.225122 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5df711-4d82-4ea4-be6f-8218eb42cf40 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:27.225398 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5df711-4d82-4ea4-be6f-8218eb42cf40 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:27.225666 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5df711-4d82-4ea4-be6f-8218eb42cf40 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:27.233329 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5df711-4d82-4ea4-be6f-8218eb42cf40 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:27.234950 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5df711-4d82-4ea4-be6f-8218eb42cf40 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:27.247225 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-7b5df711-4d82-4ea4-be6f-8218eb42cf40 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:55:27.247225 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:27.247225 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:55:27.247225 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:55:27.247225 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:55:27.247225 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:55:27.247225 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:55:27.247225 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:55:27.247225 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:55:27.247225 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:55:27.247225 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:55:27.247225 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:55:27.247225 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 16:55:27.247225 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 16:55:27.247225 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 16:55:27.248151 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 16:55:27.248151 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:55:27.248151 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:55:27.248151 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:55:27.248151 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:55:27.248151 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:55:27.248151 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:55:27.248151 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:55:27.248151 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:55:27.248151 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:55:27.248151 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:55:27.248151 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:55:27.248151 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:55:27.248151 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:55:27.248151 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:55:27.248151 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:55:27.248151 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:55:27.249075 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 16:55:27.249075 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 16:55:27.249075 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 16:55:27.249075 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 16:55:27.249075 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 16:55:27.249075 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 16:55:27.249075 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:55:27.249075 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:55:27.249075 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 538/1070] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 16:55:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 16:55:27.254442 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd663547-e35d-4272-a621-d9966d6fc291 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:27.324962 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:27.356085 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd663547-e35d-4272-a621-d9966d6fc291 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:27.390704 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7bb7c7-f753-4945-b889-e8bbc1ed22c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:27.391381 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7bb7c7-f753-4945-b889-e8bbc1ed22c7 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:27.391665 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7bb7c7-f753-4945-b889-e8bbc1ed22c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:27.391945 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7bb7c7-f753-4945-b889-e8bbc1ed22c7 None admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:27.392642 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7bb7c7-f753-4945-b889-e8bbc1ed22c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:27.392952 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7bb7c7-f753-4945-b889-e8bbc1ed22c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:27.394543 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7bb7c7-f753-4945-b889-e8bbc1ed22c7 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:27.411930 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 533/1071] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 16:55:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:55:27.413457 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5df711-4d82-4ea4-be6f-8218eb42cf40 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:27.514967 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:27.518711 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5df711-4d82-4ea4-be6f-8218eb42cf40 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:27.554609 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfb8202-58c7-442f-85dd-5a9fc45c6951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:27.555470 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb8202-58c7-442f-85dd-5a9fc45c6951 None admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:27.555753 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb8202-58c7-442f-85dd-5a9fc45c6951 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:27.556023 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb8202-58c7-442f-85dd-5a9fc45c6951 None admin] PATH_INFO: `/v3/projects/2293c0aab61f44849d0d48b214cde7ea/users/55723c756cae4e7ab636054db48de02d/roles/bae4160aa9de4d69b42641fd18c53e55` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:27.558114 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfb8202-58c7-442f-85dd-5a9fc45c6951 None admin] RBAC: Authorizing `identity:create_grant(project_id=2293c0aab61f44849d0d48b214cde7ea, user_id=55723c756cae4e7ab636054db48de02d, role_id=bae4160aa9de4d69b42641fd18c53e55)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:27.559560 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfb8202-58c7-442f-85dd-5a9fc45c6951 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:27.644320 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 539/1072] 10.222.0.82 () {58 vars in 1439 bytes} [Thu Aug 18 16:55:27 2022] PUT /identity/v3/projects/2293c0aab61f44849d0d48b214cde7ea/users/55723c756cae4e7ab636054db48de02d/roles/bae4160aa9de4d69b42641fd18c53e55 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 16:55:27.850782 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765ca7e-e24f-4117-bc75-cb1649789185 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:27.851300 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765ca7e-e24f-4117-bc75-cb1649789185 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:27.851300 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765ca7e-e24f-4117-bc75-cb1649789185 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:27.853526 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 534/1073] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:55:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:55:27.858683 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b5bed0-f960-4ee8-93cf-4da26de32824 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:27.859006 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b5bed0-f960-4ee8-93cf-4da26de32824 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:27.859289 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b5bed0-f960-4ee8-93cf-4da26de32824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:27.878071 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-d8b5bed0-f960-4ee8-93cf-4da26de32824 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:55:27.879286 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-d8b5bed0-f960-4ee8-93cf-4da26de32824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:55:27.946197 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 540/1074] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:55:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 16:55:28.216146 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5765ca7e-e24f-4117-bc75-cb1649789185 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:28.317928 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5765ca7e-e24f-4117-bc75-cb1649789185 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:28.346408 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:28.352625 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbb6105-81f9-4530-a946-d648ceaca68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:28.353255 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbb6105-81f9-4530-a946-d648ceaca68d None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:28.353537 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbb6105-81f9-4530-a946-d648ceaca68d None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:28.353811 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbb6105-81f9-4530-a946-d648ceaca68d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:28.364064 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbb6105-81f9-4530-a946-d648ceaca68d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:55:28.369275 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-cbbb6105-81f9-4530-a946-d648ceaca68d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 16:55:28.369275 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:28.369275 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:55:28.369275 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:55:28.369275 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:55:28.369275 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:55:28.369275 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:55:28.369275 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:55:28.369275 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:55:28.369275 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:55:28.369275 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:55:28.369275 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:55:28.369275 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:55:28.369275 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:55:28.369275 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:55:28.369275 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:55:28.370216 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:55:28.370216 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:55:28.370216 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:55:28.370216 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:55:28.370216 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:55:28.370216 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:55:28.370216 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:55:28.370216 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:55:28.370216 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:55:28.370216 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:55:28.370216 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:55:28.370216 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:55:28.370216 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:55:28.370216 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:55:28.370216 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:55:28.370216 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:55:28.371087 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:55:28.371087 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:55:28.371087 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:55:28.371087 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:55:28.371087 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:55:28.371087 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:55:28.371087 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:55:28.371087 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:55:28.371087 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:55:28.371087 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:55:28.371087 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 16:55:28.371087 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:55:28.375215 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 535/1075] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 16:55:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:55:28.376465 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b5bed0-f960-4ee8-93cf-4da26de32824 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:28.473587 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:28.483598 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b5bed0-f960-4ee8-93cf-4da26de32824 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:28.518260 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc321815-dd17-423d-b2f4-88a0185b7ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:28.518867 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc321815-dd17-423d-b2f4-88a0185b7ed8 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:28.519172 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc321815-dd17-423d-b2f4-88a0185b7ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:28.519443 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc321815-dd17-423d-b2f4-88a0185b7ed8 None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:28.520093 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc321815-dd17-423d-b2f4-88a0185b7ed8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:28.520394 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc321815-dd17-423d-b2f4-88a0185b7ed8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:28.521750 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc321815-dd17-423d-b2f4-88a0185b7ed8 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:28.536439 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 541/1076] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:55:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:28.537240 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbb6105-81f9-4530-a946-d648ceaca68d None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:28.639691 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbb6105-81f9-4530-a946-d648ceaca68d None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:28.673846 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72242862-d3b0-4d5d-89e6-272dc94d10b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:28.674481 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72242862-d3b0-4d5d-89e6-272dc94d10b7 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:28.674759 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72242862-d3b0-4d5d-89e6-272dc94d10b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:28.675052 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72242862-d3b0-4d5d-89e6-272dc94d10b7 None admin] PATH_INFO: `/v3/users/d93b0f69c8314968964278e6a4d18d34` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:28.676152 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72242862-d3b0-4d5d-89e6-272dc94d10b7 None admin] RBAC: Authorizing `identity:get_user(user_id=d93b0f69c8314968964278e6a4d18d34)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:28.677581 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72242862-d3b0-4d5d-89e6-272dc94d10b7 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:28.679324 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 536/1077] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 16:55:28 2022] GET /identity/v3/users/d93b0f69c8314968964278e6a4d18d34 => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:55:28.685679 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc321815-dd17-423d-b2f4-88a0185b7ed8 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:28.795985 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc321815-dd17-423d-b2f4-88a0185b7ed8 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:28.830234 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4b0fbc-9bb9-446c-81d3-bcbccafb04b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:28.830858 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4b0fbc-9bb9-446c-81d3-bcbccafb04b9 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:28.831191 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4b0fbc-9bb9-446c-81d3-bcbccafb04b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:28.831462 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4b0fbc-9bb9-446c-81d3-bcbccafb04b9 None admin] PATH_INFO: `/v3/projects/b83fb425981746e3baea3443652d4a7a` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:28.832552 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4b0fbc-9bb9-446c-81d3-bcbccafb04b9 None admin] RBAC: Authorizing `identity:get_project(project_id=b83fb425981746e3baea3443652d4a7a)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:28.833967 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4b0fbc-9bb9-446c-81d3-bcbccafb04b9 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:28.835761 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 542/1078] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 16:55:28 2022] GET /identity/v3/projects/b83fb425981746e3baea3443652d4a7a => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:28.842552 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72242862-d3b0-4d5d-89e6-272dc94d10b7 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:28.901609 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:28.954792 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72242862-d3b0-4d5d-89e6-272dc94d10b7 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:28.989073 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9f63c2-371d-4c6f-b907-aa28bd244c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:28.989741 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f63c2-371d-4c6f-b907-aa28bd244c6e None admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:28.990021 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f63c2-371d-4c6f-b907-aa28bd244c6e None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:28.990298 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f63c2-371d-4c6f-b907-aa28bd244c6e None admin] PATH_INFO: `/v3/projects/b83fb425981746e3baea3443652d4a7a/users/d93b0f69c8314968964278e6a4d18d34/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:28.992241 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9f63c2-371d-4c6f-b907-aa28bd244c6e None admin] RBAC: Authorizing `identity:create_grant(project_id=b83fb425981746e3baea3443652d4a7a, user_id=d93b0f69c8314968964278e6a4d18d34, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:28.993685 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9f63c2-371d-4c6f-b907-aa28bd244c6e None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:29.037391 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b70ae96-eb9a-47e8-8d9f-a9fea16c04f3 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:29.037695 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b70ae96-eb9a-47e8-8d9f-a9fea16c04f3 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:29.037969 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b70ae96-eb9a-47e8-8d9f-a9fea16c04f3 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:29.039969 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 543/1079] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:55:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:55:29.044999 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57109d8c-cd81-431d-8be4-35fe82ba4585 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:29.045290 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57109d8c-cd81-431d-8be4-35fe82ba4585 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:29.045569 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57109d8c-cd81-431d-8be4-35fe82ba4585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:29.064413 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-57109d8c-cd81-431d-8be4-35fe82ba4585 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:55:29.065643 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-57109d8c-cd81-431d-8be4-35fe82ba4585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:55:29.077823 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 537/1080] 10.222.0.82 () {58 vars in 1439 bytes} [Thu Aug 18 16:55:28 2022] PUT /identity/v3/projects/b83fb425981746e3baea3443652d4a7a/users/d93b0f69c8314968964278e6a4d18d34/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 16:55:29.081404 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:29.142944 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 544/1081] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:55:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 16:55:29.410973 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9f63c2-371d-4c6f-b907-aa28bd244c6e None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:29.512030 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9f63c2-371d-4c6f-b907-aa28bd244c6e None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:29.530643 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:29.548322 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cf95b4-a945-4318-9ff8-a8d0a26745ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:29.548936 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf95b4-a945-4318-9ff8-a8d0a26745ae None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:29.549040 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e76c41-0741-40bd-b3fc-8d6adc55ef1f None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:29.549218 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf95b4-a945-4318-9ff8-a8d0a26745ae None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:29.549343 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e76c41-0741-40bd-b3fc-8d6adc55ef1f None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:29.549506 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e76c41-0741-40bd-b3fc-8d6adc55ef1f None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:29.549599 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf95b4-a945-4318-9ff8-a8d0a26745ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:29.550976 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 545/1082] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:55:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 16:55:29.556208 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77054f3-4fee-484b-8a23-195bfa80765f None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:29.556499 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77054f3-4fee-484b-8a23-195bfa80765f None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:29.556773 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77054f3-4fee-484b-8a23-195bfa80765f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:29.574234 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-e77054f3-4fee-484b-8a23-195bfa80765f None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:55:29.575536 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-e77054f3-4fee-484b-8a23-195bfa80765f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:55:29.621107 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cf95b4-a945-4318-9ff8-a8d0a26745ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:29.622571 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cf95b4-a945-4318-9ff8-a8d0a26745ae None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:29.645900 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 546/1083] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:55:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 16:55:29.723482 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 538/1084] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:55:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 16:55:29.729611 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77054f3-4fee-484b-8a23-195bfa80765f None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:29.818919 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:29.838874 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77054f3-4fee-484b-8a23-195bfa80765f None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:29.872062 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3510cb5a-5f5d-4918-ba6d-cdaadd818cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:29.872684 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3510cb5a-5f5d-4918-ba6d-cdaadd818cd1 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:29.872960 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3510cb5a-5f5d-4918-ba6d-cdaadd818cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:29.873229 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3510cb5a-5f5d-4918-ba6d-cdaadd818cd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:29.883772 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3510cb5a-5f5d-4918-ba6d-cdaadd818cd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:29.885199 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3510cb5a-5f5d-4918-ba6d-cdaadd818cd1 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:29.899784 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-3510cb5a-5f5d-4918-ba6d-cdaadd818cd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:55:29.899784 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:29.899784 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:55:29.899784 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:55:29.899784 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:55:29.899784 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:55:29.899784 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:55:29.899784 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:55:29.899784 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:55:29.899784 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:55:29.899784 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:55:29.899784 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:55:29.899784 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:55:29.899784 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:55:29.899784 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:55:29.899784 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:55:29.900712 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:55:29.900712 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:55:29.900712 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:55:29.900712 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:55:29.900712 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:55:29.900712 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:55:29.900712 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:55:29.900712 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:55:29.900712 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:55:29.900712 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:55:29.900712 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:55:29.900712 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:55:29.900712 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:55:29.900712 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:55:29.900712 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:55:29.900712 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:55:29.900712 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:55:29.906736 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:55:29.906736 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:55:29.906736 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:55:29.906736 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 547/1085] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:55:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:55:29.907230 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cf95b4-a945-4318-9ff8-a8d0a26745ae None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:29.912380 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3510cb5a-5f5d-4918-ba6d-cdaadd818cd1 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:29.925816 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:30.013040 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cf95b4-a945-4318-9ff8-a8d0a26745ae None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:30.018522 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3510cb5a-5f5d-4918-ba6d-cdaadd818cd1 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:30.050077 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:30.050821 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33efd669-1839-49ca-a0fd-8f41c4f307db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:30.051485 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33efd669-1839-49ca-a0fd-8f41c4f307db None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:30.051767 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33efd669-1839-49ca-a0fd-8f41c4f307db None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:30.052040 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33efd669-1839-49ca-a0fd-8f41c4f307db None admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:30.052681 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33efd669-1839-49ca-a0fd-8f41c4f307db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:30.052982 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33efd669-1839-49ca-a0fd-8f41c4f307db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:30.054367 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33efd669-1839-49ca-a0fd-8f41c4f307db None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:30.057606 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa16e52-1b89-48f6-a540-eea144dc0904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:30.058303 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa16e52-1b89-48f6-a540-eea144dc0904 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:30.063553 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa16e52-1b89-48f6-a540-eea144dc0904 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:30.063553 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa16e52-1b89-48f6-a540-eea144dc0904 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:30.067027 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 539/1086] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:55:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:30.076584 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa16e52-1b89-48f6-a540-eea144dc0904 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:55:30.082376 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-9aa16e52-1b89-48f6-a540-eea144dc0904 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 18 16:55:30.082376 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:30.082376 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:55:30.082376 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:55:30.082376 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:55:30.082376 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:55:30.082376 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:55:30.082376 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:55:30.082376 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:55:30.082376 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:55:30.082376 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:55:30.082376 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:55:30.082376 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:55:30.082376 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:55:30.082376 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:55:30.082376 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:55:30.083652 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:55:30.083652 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:55:30.083652 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:55:30.083652 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:55:30.083652 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:55:30.083652 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:55:30.083652 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:55:30.083652 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:55:30.083652 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:55:30.083652 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:55:30.083652 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:55:30.083652 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:55:30.083652 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:55:30.083652 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:55:30.083652 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:55:30.083652 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:55:30.087608 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:55:30.087608 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:55:30.087608 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:55:30.087608 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:55:30.087608 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:55:30.087608 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:55:30.087608 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:55:30.087608 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:55:30.087608 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:55:30.087608 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:55:30.087608 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 18 16:55:30.087608 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:55:30.087608 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 548/1087] 10.222.0.82 () {56 vars in 1120 bytes} [Thu Aug 18 16:55:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 16:55:30.090290 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33efd669-1839-49ca-a0fd-8f41c4f307db None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:30.182810 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:30.205161 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33efd669-1839-49ca-a0fd-8f41c4f307db None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:30.241133 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579842ba-05ca-4363-aecd-72c134a03001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:30.242068 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579842ba-05ca-4363-aecd-72c134a03001 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:30.242347 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579842ba-05ca-4363-aecd-72c134a03001 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:30.242622 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579842ba-05ca-4363-aecd-72c134a03001 None admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:30.243466 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579842ba-05ca-4363-aecd-72c134a03001 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:30.243654 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579842ba-05ca-4363-aecd-72c134a03001 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:30.245582 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579842ba-05ca-4363-aecd-72c134a03001 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:30.253736 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 540/1088] 10.222.0.82 () {56 vars in 1115 bytes} [Thu Aug 18 16:55:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:30.259575 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa16e52-1b89-48f6-a540-eea144dc0904 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:30.373596 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa16e52-1b89-48f6-a540-eea144dc0904 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:30.407126 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b0790-aab1-4322-9ebb-182fff575e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:30.407759 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b0790-aab1-4322-9ebb-182fff575e42 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:30.408038 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b0790-aab1-4322-9ebb-182fff575e42 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:30.408312 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b0790-aab1-4322-9ebb-182fff575e42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:30.419514 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191b0790-aab1-4322-9ebb-182fff575e42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:30.420950 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191b0790-aab1-4322-9ebb-182fff575e42 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:30.435011 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-191b0790-aab1-4322-9ebb-182fff575e42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:55:30.435011 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:30.435011 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:55:30.435011 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:55:30.435011 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:55:30.435011 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:55:30.435011 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:55:30.435011 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:55:30.435011 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:55:30.435011 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:55:30.435011 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:55:30.435011 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:55:30.435011 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:55:30.435011 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:55:30.435011 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:55:30.435011 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:55:30.435931 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:55:30.435931 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:55:30.435931 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:55:30.435931 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:55:30.435931 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:55:30.435931 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:55:30.435931 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:55:30.435931 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:55:30.435931 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:55:30.435931 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:55:30.435931 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:55:30.435931 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:55:30.435931 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:55:30.435931 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:55:30.435931 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:55:30.435931 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:55:30.435931 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:55:30.436808 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:55:30.436808 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:55:30.436808 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:55:30.436808 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 549/1089] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:55:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:55:30.442944 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579842ba-05ca-4363-aecd-72c134a03001 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:30.542992 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579842ba-05ca-4363-aecd-72c134a03001 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:30.575686 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f403aaa-10de-4d81-8eef-b227c1be3216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:30.576286 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f403aaa-10de-4d81-8eef-b227c1be3216 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:30.576563 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f403aaa-10de-4d81-8eef-b227c1be3216 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:30.576830 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f403aaa-10de-4d81-8eef-b227c1be3216 None admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:30.577456 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f403aaa-10de-4d81-8eef-b227c1be3216 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:30.577755 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f403aaa-10de-4d81-8eef-b227c1be3216 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:30.579156 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f403aaa-10de-4d81-8eef-b227c1be3216 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:30.588599 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:30.591864 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 541/1090] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:55:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:30.597447 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b0790-aab1-4322-9ebb-182fff575e42 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:30.643855 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:30.697656 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b0790-aab1-4322-9ebb-182fff575e42 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:30.730609 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984c8f15-eca2-4217-b913-01463475f9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:30.731261 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c8f15-eca2-4217-b913-01463475f9da None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:30.731535 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c8f15-eca2-4217-b913-01463475f9da None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:30.731801 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c8f15-eca2-4217-b913-01463475f9da None admin] PATH_INFO: `/v3/users/nova` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:30.739105 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984c8f15-eca2-4217-b913-01463475f9da None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:30.740530 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984c8f15-eca2-4217-b913-01463475f9da None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:30.751283 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-984c8f15-eca2-4217-b913-01463475f9da None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 18 16:55:30.751283 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:30.751283 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:55:30.751283 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:55:30.751283 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:55:30.751283 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:55:30.751283 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:55:30.751283 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:55:30.751283 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:55:30.751283 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:55:30.751283 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:55:30.751283 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:55:30.751283 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 16:55:30.751283 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 16:55:30.751283 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 16:55:30.751283 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 16:55:30.752379 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:55:30.752379 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:55:30.752379 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 16:55:30.752379 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:55:30.752379 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 16:55:30.752379 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:55:30.752379 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:55:30.752379 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:55:30.752379 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:55:30.752379 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:55:30.752379 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:55:30.752379 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:55:30.752379 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:55:30.752379 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:55:30.752379 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:55:30.752379 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:55:30.752379 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:55:30.752379 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:55:30.753398 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:55:30.753398 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:55:30.753398 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 16:55:30.753398 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 16:55:30.753398 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 16:55:30.753398 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 16:55:30.753398 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 16:55:30.753398 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 16:55:30.753398 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 18 16:55:30.753398 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:55:30.753398 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 550/1091] 10.222.0.82 () {56 vars in 1093 bytes} [Thu Aug 18 16:55:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:55:30.758846 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f403aaa-10de-4d81-8eef-b227c1be3216 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:30.858133 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f403aaa-10de-4d81-8eef-b227c1be3216 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:30.891468 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7384736-6719-4b01-a685-800db854a845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:30.892072 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7384736-6719-4b01-a685-800db854a845 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:30.892347 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7384736-6719-4b01-a685-800db854a845 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:30.892616 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7384736-6719-4b01-a685-800db854a845 None admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:30.893326 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7384736-6719-4b01-a685-800db854a845 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:30.893627 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7384736-6719-4b01-a685-800db854a845 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:30.895137 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7384736-6719-4b01-a685-800db854a845 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:30.912274 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 542/1092] 10.222.0.82 () {56 vars in 1133 bytes} [Thu Aug 18 16:55:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:30.918141 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984c8f15-eca2-4217-b913-01463475f9da None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:31.005805 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7f1985-4f63-4918-8fde-9dd014964914 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:31.006103 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7f1985-4f63-4918-8fde-9dd014964914 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:31.006379 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7f1985-4f63-4918-8fde-9dd014964914 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:31.008626 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 543/1093] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:55:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 16:55:31.013688 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba685846-b367-49be-b8ab-f380b7ea0eb5 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:31.013980 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba685846-b367-49be-b8ab-f380b7ea0eb5 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:31.014310 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba685846-b367-49be-b8ab-f380b7ea0eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:31.017469 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984c8f15-eca2-4217-b913-01463475f9da None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:31.019823 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:31.031815 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-ba685846-b367-49be-b8ab-f380b7ea0eb5 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:55:31.033001 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-ba685846-b367-49be-b8ab-f380b7ea0eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:55:31.052934 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6918b368-ff54-42ea-843e-b67d304dc2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:31.053544 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6918b368-ff54-42ea-843e-b67d304dc2ff None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:31.053826 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6918b368-ff54-42ea-843e-b67d304dc2ff None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:31.054101 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6918b368-ff54-42ea-843e-b67d304dc2ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:31.091545 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:31.105774 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 544/1094] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:55:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 16:55:31.122331 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6918b368-ff54-42ea-843e-b67d304dc2ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:31.123809 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6918b368-ff54-42ea-843e-b67d304dc2ff None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:31.219148 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 551/1095] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:55:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 16:55:31.224672 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba685846-b367-49be-b8ab-f380b7ea0eb5 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:31.322277 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba685846-b367-49be-b8ab-f380b7ea0eb5 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:31.355543 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983f38df-1d5f-408d-a4bc-36713aff71c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:31.358847 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983f38df-1d5f-408d-a4bc-36713aff71c5 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:31.358847 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983f38df-1d5f-408d-a4bc-36713aff71c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:31.358847 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983f38df-1d5f-408d-a4bc-36713aff71c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:31.367041 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983f38df-1d5f-408d-a4bc-36713aff71c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:31.368654 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983f38df-1d5f-408d-a4bc-36713aff71c5 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:31.377297 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6918b368-ff54-42ea-843e-b67d304dc2ff None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:31.380741 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:31.383706 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-983f38df-1d5f-408d-a4bc-36713aff71c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:55:31.383706 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:31.383706 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:55:31.383706 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:55:31.383706 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:55:31.383706 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:55:31.383706 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:55:31.383706 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:55:31.383706 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:55:31.383706 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:55:31.383706 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:55:31.383706 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:55:31.383706 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:55:31.383706 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:55:31.383706 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:55:31.383706 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:55:31.384748 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:55:31.384748 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:55:31.384748 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:55:31.384748 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:55:31.384748 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:55:31.384748 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:55:31.384748 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:55:31.384748 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:55:31.384748 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:55:31.384748 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:55:31.384748 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:55:31.384748 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:55:31.384748 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:55:31.384748 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:55:31.384748 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:55:31.384748 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:55:31.384748 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:55:31.385701 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:55:31.385701 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:55:31.385701 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:55:31.385701 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 545/1096] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:55:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 16:55:31.390328 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983f38df-1d5f-408d-a4bc-36713aff71c5 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:31.433504 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:31.477348 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6918b368-ff54-42ea-843e-b67d304dc2ff None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:31.491085 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983f38df-1d5f-408d-a4bc-36713aff71c5 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:31.511202 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d085c3ee-5e50-4794-80e7-64211a256a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:31.511979 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d085c3ee-5e50-4794-80e7-64211a256a93 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:31.512259 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d085c3ee-5e50-4794-80e7-64211a256a93 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:31.512533 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d085c3ee-5e50-4794-80e7-64211a256a93 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:31.522708 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d085c3ee-5e50-4794-80e7-64211a256a93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:55:31.524422 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d35b73-c915-4ac7-9182-896c638e8a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:31.525013 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d35b73-c915-4ac7-9182-896c638e8a7b None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:31.525291 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d35b73-c915-4ac7-9182-896c638e8a7b None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:31.525564 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d35b73-c915-4ac7-9182-896c638e8a7b None admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:31.526188 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d35b73-c915-4ac7-9182-896c638e8a7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:31.526487 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d35b73-c915-4ac7-9182-896c638e8a7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:31.527894 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d35b73-c915-4ac7-9182-896c638e8a7b None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:31.528273 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-d085c3ee-5e50-4794-80e7-64211a256a93 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 16:55:31.528273 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:31.528273 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:55:31.528273 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:55:31.528273 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:55:31.528273 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:55:31.528273 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:55:31.528273 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:55:31.528273 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:55:31.528273 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:55:31.528273 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:55:31.528273 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:55:31.528273 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:55:31.528273 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:55:31.528273 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:55:31.528273 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:55:31.529299 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:55:31.529299 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:55:31.529299 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:55:31.529299 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:55:31.529299 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:55:31.529299 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:55:31.529299 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:55:31.529299 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:55:31.529299 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:55:31.529299 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:55:31.529299 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:55:31.529299 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:55:31.529299 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:55:31.529299 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:55:31.529299 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:55:31.529299 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:55:31.530113 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:55:31.530113 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:55:31.530113 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:55:31.530113 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:55:31.530113 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:55:31.530113 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:55:31.530113 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:55:31.530113 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:55:31.530113 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:55:31.530113 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:55:31.530113 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 16:55:31.530113 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:55:31.534692 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 552/1097] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 16:55:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 16:55:31.535558 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d085c3ee-5e50-4794-80e7-64211a256a93 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:31.539595 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 546/1098] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:55:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:55:31.544938 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d35b73-c915-4ac7-9182-896c638e8a7b None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:31.602621 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:31.639000 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:31.639162 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d085c3ee-5e50-4794-80e7-64211a256a93 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:31.641812 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d35b73-c915-4ac7-9182-896c638e8a7b None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:31.672720 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebde135-5e7e-4ff9-82c3-61bc8084d5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:31.673323 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebde135-5e7e-4ff9-82c3-61bc8084d5af None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:31.673602 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebde135-5e7e-4ff9-82c3-61bc8084d5af None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:31.673875 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebde135-5e7e-4ff9-82c3-61bc8084d5af None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:31.674523 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebde135-5e7e-4ff9-82c3-61bc8084d5af None admin] RBAC: Adding query filter params (name=admin) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:31.674827 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebde135-5e7e-4ff9-82c3-61bc8084d5af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:31.675508 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2839dd1e-9f14-4b86-b531-691475993204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:31.676125 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2839dd1e-9f14-4b86-b531-691475993204 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:31.676234 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebde135-5e7e-4ff9-82c3-61bc8084d5af None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:31.676404 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2839dd1e-9f14-4b86-b531-691475993204 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:31.676675 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2839dd1e-9f14-4b86-b531-691475993204 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:31.683838 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2839dd1e-9f14-4b86-b531-691475993204 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:31.684572 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 553/1099] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:55:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:31.685333 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2839dd1e-9f14-4b86-b531-691475993204 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:31.690325 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebde135-5e7e-4ff9-82c3-61bc8084d5af None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:31.697132 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-2839dd1e-9f14-4b86-b531-691475993204 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:55:31.697132 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:31.697132 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:55:31.697132 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:55:31.697132 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:55:31.697132 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:55:31.697132 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:55:31.697132 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:55:31.697132 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:55:31.697132 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:55:31.697132 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:55:31.697132 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:55:31.697132 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 16:55:31.697132 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 16:55:31.697132 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 16:55:31.698014 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 16:55:31.698014 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:55:31.698014 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:55:31.698014 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:55:31.698014 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:55:31.698014 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:55:31.698014 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:55:31.698014 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:55:31.698014 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:55:31.698014 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:55:31.698014 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:55:31.698014 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:55:31.698014 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:55:31.698014 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:55:31.698014 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:55:31.698014 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:55:31.698014 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:55:31.698926 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 16:55:31.698926 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 16:55:31.698926 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 16:55:31.698926 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 16:55:31.698926 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 16:55:31.698926 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 16:55:31.698926 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:55:31.698926 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:55:31.698926 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 547/1100] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 16:55:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 16:55:31.703899 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2839dd1e-9f14-4b86-b531-691475993204 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:31.797524 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebde135-5e7e-4ff9-82c3-61bc8084d5af None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:31.802581 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2839dd1e-9f14-4b86-b531-691475993204 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:31.823720 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:31.832196 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f01ade-0531-41db-bebb-b1b4904ece48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:31.832827 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f01ade-0531-41db-bebb-b1b4904ece48 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:31.833111 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f01ade-0531-41db-bebb-b1b4904ece48 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:31.833388 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f01ade-0531-41db-bebb-b1b4904ece48 None admin] PATH_INFO: `/v3/users/d93b0f69c8314968964278e6a4d18d34` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:31.834551 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f01ade-0531-41db-bebb-b1b4904ece48 None admin] RBAC: Authorizing `identity:get_user(user_id=d93b0f69c8314968964278e6a4d18d34)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:31.836471 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f01ade-0531-41db-bebb-b1b4904ece48 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:31.837555 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abcc14b-ab50-412b-92fc-ebfa09561d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:31.838066 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 554/1101] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 16:55:31 2022] GET /identity/v3/users/d93b0f69c8314968964278e6a4d18d34 => generated 285 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:55:31.838168 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abcc14b-ab50-412b-92fc-ebfa09561d9a None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:31.841859 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abcc14b-ab50-412b-92fc-ebfa09561d9a None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:31.841859 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abcc14b-ab50-412b-92fc-ebfa09561d9a None admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:31.841859 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abcc14b-ab50-412b-92fc-ebfa09561d9a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:31.841859 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abcc14b-ab50-412b-92fc-ebfa09561d9a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:31.841859 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abcc14b-ab50-412b-92fc-ebfa09561d9a None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:31.844029 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f01ade-0531-41db-bebb-b1b4904ece48 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:31.853080 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 548/1102] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 16:55:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:55:31.858733 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abcc14b-ab50-412b-92fc-ebfa09561d9a None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:31.880006 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:31.944818 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f01ade-0531-41db-bebb-b1b4904ece48 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:31.956704 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abcc14b-ab50-412b-92fc-ebfa09561d9a None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:31.978484 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d4755f-9f9d-4130-9329-07d50b70ed48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:31.979114 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d4755f-9f9d-4130-9329-07d50b70ed48 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:31.979395 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d4755f-9f9d-4130-9329-07d50b70ed48 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:31.979669 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d4755f-9f9d-4130-9329-07d50b70ed48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:31.989884 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b29ce93-8f7d-48b7-9ce9-026e88a78e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:31.990484 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b29ce93-8f7d-48b7-9ce9-026e88a78e03 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:31.990759 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b29ce93-8f7d-48b7-9ce9-026e88a78e03 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:31.991085 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b29ce93-8f7d-48b7-9ce9-026e88a78e03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:31.992365 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b29ce93-8f7d-48b7-9ce9-026e88a78e03 None admin] RBAC: Adding query filter params (role.id=bae4160aa9de4d69b42641fd18c53e55, scope.project.id=2293c0aab61f44849d0d48b214cde7ea, user.id=55723c756cae4e7ab636054db48de02d) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:31.992833 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b29ce93-8f7d-48b7-9ce9-026e88a78e03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:31.994272 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b29ce93-8f7d-48b7-9ce9-026e88a78e03 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:32.000092 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 549/1103] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 16:55:31 2022] GET /identity/v3/role_assignments?user.id=55723c756cae4e7ab636054db48de02d&scope.project.id=2293c0aab61f44849d0d48b214cde7ea&role.id=bae4160aa9de4d69b42641fd18c53e55 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:55:32.044935 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:32.052952 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d4755f-9f9d-4130-9329-07d50b70ed48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:32.054511 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d4755f-9f9d-4130-9329-07d50b70ed48 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:32.160975 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 555/1104] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:55:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 18 16:55:32.167287 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b29ce93-8f7d-48b7-9ce9-026e88a78e03 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:32.253134 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:32.269615 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b29ce93-8f7d-48b7-9ce9-026e88a78e03 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:32.304336 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc7885c-4206-4603-bcd4-009431d34e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:32.305006 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc7885c-4206-4603-bcd4-009431d34e2c None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:32.305298 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc7885c-4206-4603-bcd4-009431d34e2c None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:32.305573 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc7885c-4206-4603-bcd4-009431d34e2c None admin] PATH_INFO: `/v3/projects/b83fb425981746e3baea3443652d4a7a` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:32.306705 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc7885c-4206-4603-bcd4-009431d34e2c None admin] RBAC: Authorizing `identity:get_project(project_id=b83fb425981746e3baea3443652d4a7a)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:32.308188 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc7885c-4206-4603-bcd4-009431d34e2c None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:32.309813 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 550/1105] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 16:55:32 2022] GET /identity/v3/projects/b83fb425981746e3baea3443652d4a7a => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:32.316096 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d4755f-9f9d-4130-9329-07d50b70ed48 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:32.481629 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:32.488487 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d4755f-9f9d-4130-9329-07d50b70ed48 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:32.524118 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aea066-33e5-486e-952b-d341a4ed5a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:32.525042 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aea066-33e5-486e-952b-d341a4ed5a66 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:32.525329 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aea066-33e5-486e-952b-d341a4ed5a66 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:32.525881 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aea066-33e5-486e-952b-d341a4ed5a66 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:32.526730 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aea066-33e5-486e-952b-d341a4ed5a66 None admin] RBAC: Adding query filter params (role.id=2c6966cfb10f4d72b0242e9fb5e13b17, scope.project.id=b83fb425981746e3baea3443652d4a7a, user.id=d93b0f69c8314968964278e6a4d18d34) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:32.527470 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aea066-33e5-486e-952b-d341a4ed5a66 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:32.528908 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aea066-33e5-486e-952b-d341a4ed5a66 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:32.535001 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 556/1106] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 16:55:32 2022] GET /identity/v3/role_assignments?user.id=d93b0f69c8314968964278e6a4d18d34&scope.project.id=b83fb425981746e3baea3443652d4a7a&role.id=2c6966cfb10f4d72b0242e9fb5e13b17 => generated 608 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:34.470571 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3941a9c-98a5-44b0-9583-7fecc360d4e2 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:34.471128 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3941a9c-98a5-44b0-9583-7fecc360d4e2 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:34.471128 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3941a9c-98a5-44b0-9583-7fecc360d4e2 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:34.472462 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 551/1107] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:55:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:55:34.477720 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b2004-20b2-4755-a7c8-d00409cb7f12 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:34.478116 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b2004-20b2-4755-a7c8-d00409cb7f12 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:34.478266 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b2004-20b2-4755-a7c8-d00409cb7f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:34.496490 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-3d6b2004-20b2-4755-a7c8-d00409cb7f12 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:55:34.497666 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-3d6b2004-20b2-4755-a7c8-d00409cb7f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:55:34.561597 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 557/1108] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:55:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 16:55:34.834009 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3941a9c-98a5-44b0-9583-7fecc360d4e2 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:34.933960 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3941a9c-98a5-44b0-9583-7fecc360d4e2 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:34.967372 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839d5faa-3828-4094-a5c1-78d73c5be835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:34.967985 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d5faa-3828-4094-a5c1-78d73c5be835 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:34.968262 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d5faa-3828-4094-a5c1-78d73c5be835 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:34.968529 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d5faa-3828-4094-a5c1-78d73c5be835 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:34.969608 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839d5faa-3828-4094-a5c1-78d73c5be835 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:34.971055 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839d5faa-3828-4094-a5c1-78d73c5be835 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:34.972560 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 552/1109] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:55:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:34.978722 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6b2004-20b2-4755-a7c8-d00409cb7f12 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:35.062909 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:35.074637 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6b2004-20b2-4755-a7c8-d00409cb7f12 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:35.106944 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff584085-2f70-4c11-be07-41d66f841861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:35.107533 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff584085-2f70-4c11-be07-41d66f841861 None admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:35.107813 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff584085-2f70-4c11-be07-41d66f841861 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:35.108082 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff584085-2f70-4c11-be07-41d66f841861 None admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:35.108722 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff584085-2f70-4c11-be07-41d66f841861 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:35.110133 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff584085-2f70-4c11-be07-41d66f841861 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:35.212201 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 558/1110] 10.222.0.82 () {60 vars in 1132 bytes} [Thu Aug 18 16:55:34 2022] POST /identity/v3/users => generated 286 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 16:55:37.150909 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d52d5-f5fa-435a-9ecc-337571a1113d None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:37.151554 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d52d5-f5fa-435a-9ecc-337571a1113d None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:37.151554 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d52d5-f5fa-435a-9ecc-337571a1113d None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:37.152863 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 553/1111] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:55:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:55:37.158096 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dcc65f-0f3d-466a-97e1-b2db0cd5b2cd None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:37.158500 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dcc65f-0f3d-466a-97e1-b2db0cd5b2cd None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:37.158657 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dcc65f-0f3d-466a-97e1-b2db0cd5b2cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:37.176362 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-d9dcc65f-0f3d-466a-97e1-b2db0cd5b2cd None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:55:37.177528 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-d9dcc65f-0f3d-466a-97e1-b2db0cd5b2cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:55:37.240972 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 559/1112] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:55:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 16:55:37.507981 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0d52d5-f5fa-435a-9ecc-337571a1113d None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:37.529508 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:37.607129 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0d52d5-f5fa-435a-9ecc-337571a1113d None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:37.639946 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f752e7ca-6b3f-471e-b7ad-f98be5b6232c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:37.640571 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f752e7ca-6b3f-471e-b7ad-f98be5b6232c None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:37.640855 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f752e7ca-6b3f-471e-b7ad-f98be5b6232c None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:37.641132 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f752e7ca-6b3f-471e-b7ad-f98be5b6232c None admin] PATH_INFO: `/v3/roles/f41f6f29d9de4b1985c8c8c14f002cf2` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:37.642511 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f752e7ca-6b3f-471e-b7ad-f98be5b6232c None admin] RBAC: Authorizing `identity:get_role(role_id=f41f6f29d9de4b1985c8c8c14f002cf2)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:37.643922 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f752e7ca-6b3f-471e-b7ad-f98be5b6232c None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:37.645221 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 554/1113] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 16:55:37 2022] GET /identity/v3/roles/f41f6f29d9de4b1985c8c8c14f002cf2 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:37.651195 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dcc65f-0f3d-466a-97e1-b2db0cd5b2cd None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:37.723199 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:37.747557 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dcc65f-0f3d-466a-97e1-b2db0cd5b2cd None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:37.780979 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b882f57-b804-45fb-8fc3-c8479015c84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:37.781610 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b882f57-b804-45fb-8fc3-c8479015c84c None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:37.781889 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b882f57-b804-45fb-8fc3-c8479015c84c None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:37.782163 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b882f57-b804-45fb-8fc3-c8479015c84c None admin] PATH_INFO: `/v3/users/d0b8077e57f642709448a1d356d95bb2` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:37.796005 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b882f57-b804-45fb-8fc3-c8479015c84c None admin] RBAC: Authorizing `identity:get_user(user_id=d0b8077e57f642709448a1d356d95bb2)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:37.797439 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b882f57-b804-45fb-8fc3-c8479015c84c None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:37.799049 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 560/1114] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 16:55:37 2022] GET /identity/v3/users/d0b8077e57f642709448a1d356d95bb2 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:37.804914 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f752e7ca-6b3f-471e-b7ad-f98be5b6232c None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:37.889745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:37.901828 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f752e7ca-6b3f-471e-b7ad-f98be5b6232c None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:37.934382 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693d74bf-02b3-4c1c-a7b1-0f3e9d7c5d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:37.935002 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d74bf-02b3-4c1c-a7b1-0f3e9d7c5d2a None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:37.935286 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d74bf-02b3-4c1c-a7b1-0f3e9d7c5d2a None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:37.935559 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d74bf-02b3-4c1c-a7b1-0f3e9d7c5d2a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:38.000201 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693d74bf-02b3-4c1c-a7b1-0f3e9d7c5d2a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:38.001653 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693d74bf-02b3-4c1c-a7b1-0f3e9d7c5d2a None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:38.083614 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:38.096476 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 555/1115] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:55:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 16:55:38.102965 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b882f57-b804-45fb-8fc3-c8479015c84c None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:38.197236 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b882f57-b804-45fb-8fc3-c8479015c84c None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:38.225446 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:38.230261 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d1136-3537-4975-a3a7-48fc8841f070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:38.230874 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d1136-3537-4975-a3a7-48fc8841f070 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:38.231180 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d1136-3537-4975-a3a7-48fc8841f070 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:38.231449 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d1136-3537-4975-a3a7-48fc8841f070 None admin] PATH_INFO: `/v3/projects/b83fb425981746e3baea3443652d4a7a` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:38.232552 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690d1136-3537-4975-a3a7-48fc8841f070 None admin] RBAC: Authorizing `identity:get_project(project_id=b83fb425981746e3baea3443652d4a7a)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:38.233983 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690d1136-3537-4975-a3a7-48fc8841f070 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:38.235589 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 561/1116] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 16:55:38 2022] GET /identity/v3/projects/b83fb425981746e3baea3443652d4a7a => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:38.241681 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693d74bf-02b3-4c1c-a7b1-0f3e9d7c5d2a None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:38.337833 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693d74bf-02b3-4c1c-a7b1-0f3e9d7c5d2a None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:38.369702 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ed7a32-4f78-49d0-a8a7-6389fbce720a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:38.370294 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed7a32-4f78-49d0-a8a7-6389fbce720a None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:38.370578 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed7a32-4f78-49d0-a8a7-6389fbce720a None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:38.370851 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed7a32-4f78-49d0-a8a7-6389fbce720a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:38.371587 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ed7a32-4f78-49d0-a8a7-6389fbce720a None admin] RBAC: Adding query filter params (role.id=f41f6f29d9de4b1985c8c8c14f002cf2, scope.project.id=b83fb425981746e3baea3443652d4a7a, user.id=d0b8077e57f642709448a1d356d95bb2) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:38.371894 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ed7a32-4f78-49d0-a8a7-6389fbce720a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:38.373315 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ed7a32-4f78-49d0-a8a7-6389fbce720a None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:38.378439 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 556/1117] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 16:55:38 2022] GET /identity/v3/role_assignments?user.id=d0b8077e57f642709448a1d356d95bb2&scope.project.id=b83fb425981746e3baea3443652d4a7a&role.id=f41f6f29d9de4b1985c8c8c14f002cf2 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:40.249122 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc40ea5-6479-4bef-9ad2-bdfda33741e4 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:40.249707 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc40ea5-6479-4bef-9ad2-bdfda33741e4 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:40.249707 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc40ea5-6479-4bef-9ad2-bdfda33741e4 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:40.251028 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 562/1118] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:55:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:55:40.256105 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834453a8-8553-4f1d-8a45-a73a7f399a85 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:40.256491 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834453a8-8553-4f1d-8a45-a73a7f399a85 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:40.256680 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834453a8-8553-4f1d-8a45-a73a7f399a85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:40.274824 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-834453a8-8553-4f1d-8a45-a73a7f399a85 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:55:40.276035 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-834453a8-8553-4f1d-8a45-a73a7f399a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:55:40.322483 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:40.341461 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 557/1119] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:55:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 16:55:40.610978 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc40ea5-6479-4bef-9ad2-bdfda33741e4 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:40.709115 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc40ea5-6479-4bef-9ad2-bdfda33741e4 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:40.742920 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3447f888-de65-47e7-80b5-da1e3d145309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:40.743544 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3447f888-de65-47e7-80b5-da1e3d145309 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:40.743824 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3447f888-de65-47e7-80b5-da1e3d145309 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:40.744098 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3447f888-de65-47e7-80b5-da1e3d145309 None admin] PATH_INFO: `/v3/roles/f41f6f29d9de4b1985c8c8c14f002cf2` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:40.745480 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3447f888-de65-47e7-80b5-da1e3d145309 None admin] RBAC: Authorizing `identity:get_role(role_id=f41f6f29d9de4b1985c8c8c14f002cf2)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:40.746888 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3447f888-de65-47e7-80b5-da1e3d145309 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:40.748214 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 563/1120] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 16:55:40 2022] GET /identity/v3/roles/f41f6f29d9de4b1985c8c8c14f002cf2 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:40.754116 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834453a8-8553-4f1d-8a45-a73a7f399a85 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:40.851758 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834453a8-8553-4f1d-8a45-a73a7f399a85 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:40.884220 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376a979c-4db6-42f6-9afa-ec30ef658eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:40.884792 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a979c-4db6-42f6-9afa-ec30ef658eff None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:40.885075 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a979c-4db6-42f6-9afa-ec30ef658eff None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:40.885352 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a979c-4db6-42f6-9afa-ec30ef658eff None admin] PATH_INFO: `/v3/users/d0b8077e57f642709448a1d356d95bb2` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:40.886452 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376a979c-4db6-42f6-9afa-ec30ef658eff None admin] RBAC: Authorizing `identity:get_user(user_id=d0b8077e57f642709448a1d356d95bb2)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:40.887927 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376a979c-4db6-42f6-9afa-ec30ef658eff None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:40.889487 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 558/1121] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 16:55:40 2022] GET /identity/v3/users/d0b8077e57f642709448a1d356d95bb2 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:40.895240 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3447f888-de65-47e7-80b5-da1e3d145309 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:40.959262 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:40.992239 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3447f888-de65-47e7-80b5-da1e3d145309 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:41.024892 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fdd728-71b1-47cc-8a91-5bd7d650e933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:41.025508 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fdd728-71b1-47cc-8a91-5bd7d650e933 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:41.025788 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fdd728-71b1-47cc-8a91-5bd7d650e933 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:41.026062 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fdd728-71b1-47cc-8a91-5bd7d650e933 None admin] PATH_INFO: `/v3/projects/b83fb425981746e3baea3443652d4a7a` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:41.027183 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fdd728-71b1-47cc-8a91-5bd7d650e933 None admin] RBAC: Authorizing `identity:get_project(project_id=b83fb425981746e3baea3443652d4a7a)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:41.028600 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fdd728-71b1-47cc-8a91-5bd7d650e933 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:41.030198 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 564/1122] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 16:55:40 2022] GET /identity/v3/projects/b83fb425981746e3baea3443652d4a7a => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:41.036067 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376a979c-4db6-42f6-9afa-ec30ef658eff None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:41.086221 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:41.134389 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376a979c-4db6-42f6-9afa-ec30ef658eff None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:41.167183 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc32b8d3-77de-4d5c-90fc-2078e2fd8341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:41.167833 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc32b8d3-77de-4d5c-90fc-2078e2fd8341 None admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:41.168117 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc32b8d3-77de-4d5c-90fc-2078e2fd8341 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:41.168394 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc32b8d3-77de-4d5c-90fc-2078e2fd8341 None admin] PATH_INFO: `/v3/projects/b83fb425981746e3baea3443652d4a7a/users/d0b8077e57f642709448a1d356d95bb2/roles/f41f6f29d9de4b1985c8c8c14f002cf2` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:41.170506 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc32b8d3-77de-4d5c-90fc-2078e2fd8341 None admin] RBAC: Authorizing `identity:create_grant(project_id=b83fb425981746e3baea3443652d4a7a, user_id=d0b8077e57f642709448a1d356d95bb2, role_id=f41f6f29d9de4b1985c8c8c14f002cf2)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:41.171984 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc32b8d3-77de-4d5c-90fc-2078e2fd8341 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:41.236321 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 559/1123] 10.222.0.82 () {58 vars in 1439 bytes} [Thu Aug 18 16:55:41 2022] PUT /identity/v3/projects/b83fb425981746e3baea3443652d4a7a/users/d0b8077e57f642709448a1d356d95bb2/roles/f41f6f29d9de4b1985c8c8c14f002cf2 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 16:55:43.095649 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f1dd96-dc7b-419e-ae6b-f116e0d7cff6 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:43.096194 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f1dd96-dc7b-419e-ae6b-f116e0d7cff6 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:43.096194 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f1dd96-dc7b-419e-ae6b-f116e0d7cff6 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:43.097511 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 565/1124] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:55:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:55:43.102641 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7db58-0637-4fb1-9073-cca3293aea2f None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:43.103055 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7db58-0637-4fb1-9073-cca3293aea2f None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:43.103304 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7db58-0637-4fb1-9073-cca3293aea2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:43.122058 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-a4a7db58-0637-4fb1-9073-cca3293aea2f None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:55:43.123269 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-a4a7db58-0637-4fb1-9073-cca3293aea2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:55:43.187258 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 560/1125] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:55:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 16:55:43.457191 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f1dd96-dc7b-419e-ae6b-f116e0d7cff6 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:43.555173 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f1dd96-dc7b-419e-ae6b-f116e0d7cff6 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:43.587203 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d4441a-4114-4137-a192-9dfcd8da1d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:43.587810 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d4441a-4114-4137-a192-9dfcd8da1d90 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:43.588082 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d4441a-4114-4137-a192-9dfcd8da1d90 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:43.588348 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d4441a-4114-4137-a192-9dfcd8da1d90 None admin] PATH_INFO: `/v3/roles/f41f6f29d9de4b1985c8c8c14f002cf2` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:43.589704 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d4441a-4114-4137-a192-9dfcd8da1d90 None admin] RBAC: Authorizing `identity:get_role(role_id=f41f6f29d9de4b1985c8c8c14f002cf2)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:43.591079 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d4441a-4114-4137-a192-9dfcd8da1d90 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:43.592365 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 566/1126] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 16:55:43 2022] GET /identity/v3/roles/f41f6f29d9de4b1985c8c8c14f002cf2 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:43.598117 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a7db58-0637-4fb1-9073-cca3293aea2f None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:43.638430 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:43.694520 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a7db58-0637-4fb1-9073-cca3293aea2f None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:43.728402 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea72e6ab-c4d4-41a2-84ea-282500393ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:43.728965 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea72e6ab-c4d4-41a2-84ea-282500393ba6 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:43.729245 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea72e6ab-c4d4-41a2-84ea-282500393ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:43.729517 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea72e6ab-c4d4-41a2-84ea-282500393ba6 None admin] PATH_INFO: `/v3/users/d0b8077e57f642709448a1d356d95bb2` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:43.730613 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea72e6ab-c4d4-41a2-84ea-282500393ba6 None admin] RBAC: Authorizing `identity:get_user(user_id=d0b8077e57f642709448a1d356d95bb2)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:43.732084 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea72e6ab-c4d4-41a2-84ea-282500393ba6 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:43.733628 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 561/1127] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 16:55:43 2022] GET /identity/v3/users/d0b8077e57f642709448a1d356d95bb2 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:43.739438 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d4441a-4114-4137-a192-9dfcd8da1d90 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:43.743361 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:43.836378 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d4441a-4114-4137-a192-9dfcd8da1d90 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:43.868521 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb4821e-1ea8-4cd3-b0a2-d41c51dddfe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:43.869118 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb4821e-1ea8-4cd3-b0a2-d41c51dddfe6 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:43.869394 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb4821e-1ea8-4cd3-b0a2-d41c51dddfe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:43.869663 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb4821e-1ea8-4cd3-b0a2-d41c51dddfe6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:43.933800 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb4821e-1ea8-4cd3-b0a2-d41c51dddfe6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:43.935246 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb4821e-1ea8-4cd3-b0a2-d41c51dddfe6 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:43.937722 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:44.028959 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 567/1128] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:55:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 16:55:44.034951 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea72e6ab-c4d4-41a2-84ea-282500393ba6 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:44.133792 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea72e6ab-c4d4-41a2-84ea-282500393ba6 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:44.142696 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:44.167451 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c8a65c-ec50-4222-bf7a-c3e84f9a3311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:44.168114 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c8a65c-ec50-4222-bf7a-c3e84f9a3311 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:44.168426 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c8a65c-ec50-4222-bf7a-c3e84f9a3311 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:44.168703 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c8a65c-ec50-4222-bf7a-c3e84f9a3311 None admin] PATH_INFO: `/v3/projects/b83fb425981746e3baea3443652d4a7a` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:44.169786 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c8a65c-ec50-4222-bf7a-c3e84f9a3311 None admin] RBAC: Authorizing `identity:get_project(project_id=b83fb425981746e3baea3443652d4a7a)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:44.171251 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c8a65c-ec50-4222-bf7a-c3e84f9a3311 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:44.172833 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 562/1129] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 16:55:44 2022] GET /identity/v3/projects/b83fb425981746e3baea3443652d4a7a => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:44.179269 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb4821e-1ea8-4cd3-b0a2-d41c51dddfe6 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:44.275330 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb4821e-1ea8-4cd3-b0a2-d41c51dddfe6 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:44.277573 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:44.307899 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f984b76a-cfe4-4136-b744-3e524d2e3217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:44.308495 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984b76a-cfe4-4136-b744-3e524d2e3217 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:44.308778 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984b76a-cfe4-4136-b744-3e524d2e3217 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:44.309053 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984b76a-cfe4-4136-b744-3e524d2e3217 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:44.309774 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f984b76a-cfe4-4136-b744-3e524d2e3217 None admin] RBAC: Adding query filter params (role.id=f41f6f29d9de4b1985c8c8c14f002cf2, scope.project.id=b83fb425981746e3baea3443652d4a7a, user.id=d0b8077e57f642709448a1d356d95bb2) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:44.310084 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f984b76a-cfe4-4136-b744-3e524d2e3217 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:44.311538 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f984b76a-cfe4-4136-b744-3e524d2e3217 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:44.317156 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 568/1130] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 16:55:44 2022] GET /identity/v3/role_assignments?user.id=d0b8077e57f642709448a1d356d95bb2&scope.project.id=b83fb425981746e3baea3443652d4a7a&role.id=f41f6f29d9de4b1985c8c8c14f002cf2 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:46.236408 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f4bf97-0611-4aee-b157-cb8ea317f146 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:46.236949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f4bf97-0611-4aee-b157-cb8ea317f146 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:46.236949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f4bf97-0611-4aee-b157-cb8ea317f146 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:46.238291 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 563/1131] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:55:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:55:46.243809 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed157370-156c-40d0-b3c5-dc08ecfaa38b None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:46.244213 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed157370-156c-40d0-b3c5-dc08ecfaa38b None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:46.244370 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed157370-156c-40d0-b3c5-dc08ecfaa38b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:46.262997 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-ed157370-156c-40d0-b3c5-dc08ecfaa38b None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:55:46.264178 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-ed157370-156c-40d0-b3c5-dc08ecfaa38b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:55:46.328729 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 569/1132] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:55:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 16:55:46.606517 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f4bf97-0611-4aee-b157-cb8ea317f146 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:46.704874 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f4bf97-0611-4aee-b157-cb8ea317f146 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:46.737902 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dceff9b-0073-464a-bcca-2a0f74cf9a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:46.738510 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dceff9b-0073-464a-bcca-2a0f74cf9a01 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:46.738787 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dceff9b-0073-464a-bcca-2a0f74cf9a01 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:46.739082 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dceff9b-0073-464a-bcca-2a0f74cf9a01 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:46.740165 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dceff9b-0073-464a-bcca-2a0f74cf9a01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:46.742490 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dceff9b-0073-464a-bcca-2a0f74cf9a01 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:46.744024 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 564/1133] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:55:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:46.749724 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed157370-156c-40d0-b3c5-dc08ecfaa38b None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:46.842509 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:46.846233 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed157370-156c-40d0-b3c5-dc08ecfaa38b None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:46.878594 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6888ae7d-961b-40d3-baa7-4a3e919e388a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:46.879170 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6888ae7d-961b-40d3-baa7-4a3e919e388a None admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:46.879450 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6888ae7d-961b-40d3-baa7-4a3e919e388a None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:46.879722 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6888ae7d-961b-40d3-baa7-4a3e919e388a None admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:46.880348 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6888ae7d-961b-40d3-baa7-4a3e919e388a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:46.882371 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6888ae7d-961b-40d3-baa7-4a3e919e388a None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:46.982882 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 570/1134] 10.222.0.82 () {60 vars in 1141 bytes} [Thu Aug 18 16:55:46 2022] POST /identity/v3/projects => generated 311 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 16:55:48.925994 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54490578-08c5-4d52-806f-a920c0d9a467 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:48.926535 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54490578-08c5-4d52-806f-a920c0d9a467 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:48.926535 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54490578-08c5-4d52-806f-a920c0d9a467 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:48.928459 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 565/1135] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:55:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:55:48.933726 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4247426-f4b0-4fd9-a0ad-ad5b62f1044d None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:48.934111 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4247426-f4b0-4fd9-a0ad-ad5b62f1044d None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:48.934287 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4247426-f4b0-4fd9-a0ad-ad5b62f1044d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:48.952588 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-a4247426-f4b0-4fd9-a0ad-ad5b62f1044d None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:55:48.953781 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-a4247426-f4b0-4fd9-a0ad-ad5b62f1044d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:55:49.019271 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 571/1136] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:55:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 16:55:49.291185 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54490578-08c5-4d52-806f-a920c0d9a467 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:49.337073 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:49.389452 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54490578-08c5-4d52-806f-a920c0d9a467 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:49.422013 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5c5b1a-db27-4273-bba3-b66a2f560272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:49.422632 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c5b1a-db27-4273-bba3-b66a2f560272 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:49.422927 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c5b1a-db27-4273-bba3-b66a2f560272 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:49.423211 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c5b1a-db27-4273-bba3-b66a2f560272 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:49.424314 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5c5b1a-db27-4273-bba3-b66a2f560272 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:49.425764 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5c5b1a-db27-4273-bba3-b66a2f560272 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:49.427323 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 566/1137] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:55:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:49.433695 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4247426-f4b0-4fd9-a0ad-ad5b62f1044d None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:49.512149 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:49.530518 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4247426-f4b0-4fd9-a0ad-ad5b62f1044d None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:49.564396 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98446582-46f0-4635-b88b-1fd497a93a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:49.564930 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98446582-46f0-4635-b88b-1fd497a93a8d None admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:49.565202 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98446582-46f0-4635-b88b-1fd497a93a8d None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:49.565470 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98446582-46f0-4635-b88b-1fd497a93a8d None admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:49.566099 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98446582-46f0-4635-b88b-1fd497a93a8d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:49.567624 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98446582-46f0-4635-b88b-1fd497a93a8d None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:49.633251 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 572/1138] 10.222.0.82 () {60 vars in 1132 bytes} [Thu Aug 18 16:55:49 2022] POST /identity/v3/users => generated 286 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 16:55:51.558625 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eced671-3431-4d84-8036-4c5e45dcb4c1 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:51.559277 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eced671-3431-4d84-8036-4c5e45dcb4c1 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:51.559277 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eced671-3431-4d84-8036-4c5e45dcb4c1 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:51.560542 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 567/1139] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:55:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:55:51.565780 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d043ba-7e87-4d4b-8a2a-2b6217254af0 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:51.566192 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d043ba-7e87-4d4b-8a2a-2b6217254af0 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:51.566336 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d043ba-7e87-4d4b-8a2a-2b6217254af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:51.585530 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-46d043ba-7e87-4d4b-8a2a-2b6217254af0 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:55:51.586711 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-46d043ba-7e87-4d4b-8a2a-2b6217254af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:55:51.651037 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 573/1140] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:55:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 16:55:51.921439 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eced671-3431-4d84-8036-4c5e45dcb4c1 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:52.020416 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eced671-3431-4d84-8036-4c5e45dcb4c1 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:52.033040 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:52.053091 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0e8e98-3773-4193-8128-b3198af54e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:52.054412 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0e8e98-3773-4193-8128-b3198af54e3f None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:52.054696 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0e8e98-3773-4193-8128-b3198af54e3f None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:52.055043 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0e8e98-3773-4193-8128-b3198af54e3f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:52.064894 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0e8e98-3773-4193-8128-b3198af54e3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:55:52.070502 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-2e0e8e98-3773-4193-8128-b3198af54e3f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 16:55:52.070502 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:52.070502 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:55:52.070502 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:55:52.070502 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:55:52.070502 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:55:52.070502 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:55:52.070502 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:55:52.070502 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:55:52.070502 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:55:52.070502 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:55:52.070502 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:55:52.070502 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:55:52.070502 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:55:52.070502 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:55:52.070502 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:55:52.071717 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:55:52.071717 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:55:52.071717 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:55:52.071717 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:55:52.071717 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:55:52.071717 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:55:52.071717 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:55:52.071717 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:55:52.071717 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:55:52.071717 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:55:52.071717 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:55:52.071717 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:55:52.071717 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:55:52.071717 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:55:52.071717 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:55:52.071717 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:55:52.072757 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:55:52.072757 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:55:52.072757 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:55:52.072757 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:55:52.072757 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:55:52.072757 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:55:52.072757 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:55:52.072757 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:55:52.072757 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:55:52.072757 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:55:52.072757 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 16:55:52.072757 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:55:52.072757 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 568/1141] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 16:55:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:55:52.077881 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d043ba-7e87-4d4b-8a2a-2b6217254af0 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:52.141040 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:52.175490 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d043ba-7e87-4d4b-8a2a-2b6217254af0 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:52.208145 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1587515c-7475-4c19-b184-c729bd01560e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:52.208740 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1587515c-7475-4c19-b184-c729bd01560e None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:52.209017 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1587515c-7475-4c19-b184-c729bd01560e None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:52.209286 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1587515c-7475-4c19-b184-c729bd01560e None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:52.211076 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1587515c-7475-4c19-b184-c729bd01560e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:52.211451 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1587515c-7475-4c19-b184-c729bd01560e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:52.212835 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1587515c-7475-4c19-b184-c729bd01560e None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:52.220864 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 574/1142] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:55:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:52.227369 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0e8e98-3773-4193-8128-b3198af54e3f None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:52.323478 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0e8e98-3773-4193-8128-b3198af54e3f None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:52.356231 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81caf0d0-7805-42fe-b47b-2bf92aab036c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:52.356847 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81caf0d0-7805-42fe-b47b-2bf92aab036c None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:52.357421 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81caf0d0-7805-42fe-b47b-2bf92aab036c None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:52.357421 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81caf0d0-7805-42fe-b47b-2bf92aab036c None admin] PATH_INFO: `/v3/users/347296c34afd4772a2a8ec78cbc4437e` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:52.369772 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81caf0d0-7805-42fe-b47b-2bf92aab036c None admin] RBAC: Authorizing `identity:get_user(user_id=347296c34afd4772a2a8ec78cbc4437e)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:52.371237 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81caf0d0-7805-42fe-b47b-2bf92aab036c None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:52.372792 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 569/1143] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 16:55:52 2022] GET /identity/v3/users/347296c34afd4772a2a8ec78cbc4437e => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:52.378649 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1587515c-7475-4c19-b184-c729bd01560e None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:52.476561 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1587515c-7475-4c19-b184-c729bd01560e None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:52.500959 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:52.508800 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78829f15-6a66-468b-ad4f-e9f3f7e32e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:52.509397 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78829f15-6a66-468b-ad4f-e9f3f7e32e70 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:52.512049 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78829f15-6a66-468b-ad4f-e9f3f7e32e70 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:52.512049 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78829f15-6a66-468b-ad4f-e9f3f7e32e70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:52.574499 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78829f15-6a66-468b-ad4f-e9f3f7e32e70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:52.576002 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78829f15-6a66-468b-ad4f-e9f3f7e32e70 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:52.675577 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 575/1144] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:55:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 16:55:52.682169 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81caf0d0-7805-42fe-b47b-2bf92aab036c None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:52.712968 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:52.787046 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81caf0d0-7805-42fe-b47b-2bf92aab036c None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:52.822108 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c12066-3d3f-4a5d-9dd3-4d288f621d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:52.822747 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c12066-3d3f-4a5d-9dd3-4d288f621d00 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:52.823068 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c12066-3d3f-4a5d-9dd3-4d288f621d00 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:52.823341 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c12066-3d3f-4a5d-9dd3-4d288f621d00 None admin] PATH_INFO: `/v3/projects/bdce85a7c68c4c9895fff4a356bce0df` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:52.824431 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c12066-3d3f-4a5d-9dd3-4d288f621d00 None admin] RBAC: Authorizing `identity:get_project(project_id=bdce85a7c68c4c9895fff4a356bce0df)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:52.825865 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c12066-3d3f-4a5d-9dd3-4d288f621d00 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:52.827494 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 570/1145] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 16:55:52 2022] GET /identity/v3/projects/bdce85a7c68c4c9895fff4a356bce0df => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:52.833793 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78829f15-6a66-468b-ad4f-e9f3f7e32e70 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:52.860777 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:52.930313 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78829f15-6a66-468b-ad4f-e9f3f7e32e70 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:52.963098 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8b6eeb-89d1-4f9e-84df-4f73b928e07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:52.963695 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8b6eeb-89d1-4f9e-84df-4f73b928e07e None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:52.963973 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8b6eeb-89d1-4f9e-84df-4f73b928e07e None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:52.964244 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8b6eeb-89d1-4f9e-84df-4f73b928e07e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:52.964962 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8b6eeb-89d1-4f9e-84df-4f73b928e07e None admin] RBAC: Adding query filter params (role.id=2c6966cfb10f4d72b0242e9fb5e13b17, scope.project.id=bdce85a7c68c4c9895fff4a356bce0df, user.id=347296c34afd4772a2a8ec78cbc4437e) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:52.965267 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8b6eeb-89d1-4f9e-84df-4f73b928e07e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:52.966677 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8b6eeb-89d1-4f9e-84df-4f73b928e07e None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:52.972012 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 576/1146] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 16:55:52 2022] GET /identity/v3/role_assignments?user.id=347296c34afd4772a2a8ec78cbc4437e&scope.project.id=bdce85a7c68c4c9895fff4a356bce0df&role.id=2c6966cfb10f4d72b0242e9fb5e13b17 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:54.826328 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d50232d-c6d3-4dde-b668-1cf2d730a928 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:54.826835 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d50232d-c6d3-4dde-b668-1cf2d730a928 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:54.826981 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d50232d-c6d3-4dde-b668-1cf2d730a928 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:54.828255 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 571/1147] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:55:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:55:54.833367 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df5e751-8ae4-4997-9863-48214c99825c None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:54.833798 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df5e751-8ae4-4997-9863-48214c99825c None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:54.833915 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df5e751-8ae4-4997-9863-48214c99825c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:54.852335 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-7df5e751-8ae4-4997-9863-48214c99825c None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:55:54.853509 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-7df5e751-8ae4-4997-9863-48214c99825c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:55:54.917232 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 577/1148] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:55:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 16:55:55.187188 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d50232d-c6d3-4dde-b668-1cf2d730a928 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:55.282570 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:55.287160 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d50232d-c6d3-4dde-b668-1cf2d730a928 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:55.322138 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9277a18-8ef7-4faf-aa9a-3c54edd88f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:55.322772 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9277a18-8ef7-4faf-aa9a-3c54edd88f4d None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:55.323395 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9277a18-8ef7-4faf-aa9a-3c54edd88f4d None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:55.323674 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9277a18-8ef7-4faf-aa9a-3c54edd88f4d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:55.333532 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9277a18-8ef7-4faf-aa9a-3c54edd88f4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:55:55.339070 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-a9277a18-8ef7-4faf-aa9a-3c54edd88f4d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 16:55:55.339070 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:55.339070 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:55:55.339070 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:55:55.339070 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:55:55.339070 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:55:55.339070 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:55:55.339070 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:55:55.339070 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:55:55.339070 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:55:55.339070 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:55:55.339070 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:55:55.339070 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:55:55.339070 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:55:55.339070 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:55:55.339070 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:55:55.340156 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:55:55.340156 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:55:55.340156 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:55:55.340156 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:55:55.340156 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:55:55.340156 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:55:55.340156 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:55:55.340156 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:55:55.340156 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:55:55.340156 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:55:55.340156 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:55:55.340156 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:55:55.340156 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:55:55.340156 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:55:55.340156 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:55:55.340156 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:55:55.341167 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:55:55.341167 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:55:55.341167 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:55:55.341167 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:55:55.341167 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:55:55.341167 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:55:55.341167 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:55:55.341167 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:55:55.341167 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:55:55.341167 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:55:55.341167 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 16:55:55.341167 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:55:55.341167 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 572/1149] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 16:55:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:55:55.346290 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df5e751-8ae4-4997-9863-48214c99825c None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:55.369485 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:55.441505 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df5e751-8ae4-4997-9863-48214c99825c None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:55.474795 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3339aeb-05f8-4d43-bb2b-2643fe2caa0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:55.475446 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3339aeb-05f8-4d43-bb2b-2643fe2caa0c None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:55.475721 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3339aeb-05f8-4d43-bb2b-2643fe2caa0c None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:55.475991 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3339aeb-05f8-4d43-bb2b-2643fe2caa0c None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:55.476639 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3339aeb-05f8-4d43-bb2b-2643fe2caa0c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:55.476940 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3339aeb-05f8-4d43-bb2b-2643fe2caa0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:55.478293 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3339aeb-05f8-4d43-bb2b-2643fe2caa0c None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:55.485924 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 578/1150] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:55:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:55.491944 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9277a18-8ef7-4faf-aa9a-3c54edd88f4d None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:55.589846 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9277a18-8ef7-4faf-aa9a-3c54edd88f4d None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:55.621680 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e8b7ba-71fa-494b-8009-4a2375edd54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:55.622295 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8b7ba-71fa-494b-8009-4a2375edd54f None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:55.622571 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8b7ba-71fa-494b-8009-4a2375edd54f None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:55.622840 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8b7ba-71fa-494b-8009-4a2375edd54f None admin] PATH_INFO: `/v3/users/347296c34afd4772a2a8ec78cbc4437e` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:55.623957 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e8b7ba-71fa-494b-8009-4a2375edd54f None admin] RBAC: Authorizing `identity:get_user(user_id=347296c34afd4772a2a8ec78cbc4437e)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:55.625388 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e8b7ba-71fa-494b-8009-4a2375edd54f None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:55.626933 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 573/1151] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 16:55:55 2022] GET /identity/v3/users/347296c34afd4772a2a8ec78cbc4437e => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:55:55.633092 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3339aeb-05f8-4d43-bb2b-2643fe2caa0c None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:55.715194 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:55.730038 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3339aeb-05f8-4d43-bb2b-2643fe2caa0c None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:55.763154 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b67e3c4-3b3b-43fe-b498-b68d2700d3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:55.763781 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b67e3c4-3b3b-43fe-b498-b68d2700d3a6 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:55.764059 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b67e3c4-3b3b-43fe-b498-b68d2700d3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:55.764329 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b67e3c4-3b3b-43fe-b498-b68d2700d3a6 None admin] PATH_INFO: `/v3/projects/bdce85a7c68c4c9895fff4a356bce0df` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:55.765440 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b67e3c4-3b3b-43fe-b498-b68d2700d3a6 None admin] RBAC: Authorizing `identity:get_project(project_id=bdce85a7c68c4c9895fff4a356bce0df)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:55.767236 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b67e3c4-3b3b-43fe-b498-b68d2700d3a6 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:55.768491 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 579/1152] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 16:55:55 2022] GET /identity/v3/projects/bdce85a7c68c4c9895fff4a356bce0df => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:55.774587 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e8b7ba-71fa-494b-8009-4a2375edd54f None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:55.801314 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:55.872340 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e8b7ba-71fa-494b-8009-4a2375edd54f None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:55.904561 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a741a4-ecc4-4a47-b417-0de724811cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:55.905212 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a741a4-ecc4-4a47-b417-0de724811cfd None admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:55.905495 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a741a4-ecc4-4a47-b417-0de724811cfd None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:55.905772 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a741a4-ecc4-4a47-b417-0de724811cfd None admin] PATH_INFO: `/v3/projects/bdce85a7c68c4c9895fff4a356bce0df/users/347296c34afd4772a2a8ec78cbc4437e/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:55.907685 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a741a4-ecc4-4a47-b417-0de724811cfd None admin] RBAC: Authorizing `identity:create_grant(project_id=bdce85a7c68c4c9895fff4a356bce0df, user_id=347296c34afd4772a2a8ec78cbc4437e, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:55.909132 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a741a4-ecc4-4a47-b417-0de724811cfd None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:56.004796 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 574/1153] 10.222.0.82 () {58 vars in 1439 bytes} [Thu Aug 18 16:55:55 2022] PUT /identity/v3/projects/bdce85a7c68c4c9895fff4a356bce0df/users/347296c34afd4772a2a8ec78cbc4437e/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 16:55:57.867437 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff73bee5-fc8b-4e5e-9f76-50ef847f848a None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:57.867957 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff73bee5-fc8b-4e5e-9f76-50ef847f848a None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:57.867957 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff73bee5-fc8b-4e5e-9f76-50ef847f848a None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:57.869312 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 580/1154] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:55:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:55:57.874533 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0babe-0b87-4b51-9f65-dd2488876114 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:57.874988 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0babe-0b87-4b51-9f65-dd2488876114 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:57.875119 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0babe-0b87-4b51-9f65-dd2488876114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:57.892808 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-5eb0babe-0b87-4b51-9f65-dd2488876114 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:55:57.894018 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-5eb0babe-0b87-4b51-9f65-dd2488876114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:55:57.958021 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 575/1155] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:55:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 16:55:58.228112 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff73bee5-fc8b-4e5e-9f76-50ef847f848a None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:58.329164 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff73bee5-fc8b-4e5e-9f76-50ef847f848a None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:58.362120 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e00626-9e85-43ae-ad3c-8081822c3aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:58.362733 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e00626-9e85-43ae-ad3c-8081822c3aad None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:58.363042 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e00626-9e85-43ae-ad3c-8081822c3aad None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:58.366929 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e00626-9e85-43ae-ad3c-8081822c3aad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:58.373191 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e00626-9e85-43ae-ad3c-8081822c3aad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:55:58.377825 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-94e00626-9e85-43ae-ad3c-8081822c3aad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 16:55:58.377825 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:55:58.377825 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:55:58.377825 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:55:58.377825 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:55:58.377825 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:55:58.377825 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:55:58.377825 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:55:58.377825 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:55:58.377825 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:55:58.377825 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:55:58.377825 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:55:58.377825 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:55:58.377825 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:55:58.377825 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:55:58.377825 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:55:58.379078 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:55:58.379078 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:55:58.379078 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:55:58.379078 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:55:58.379078 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:55:58.379078 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:55:58.379078 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:55:58.379078 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:55:58.379078 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:55:58.379078 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:55:58.379078 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:55:58.379078 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:55:58.379078 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:55:58.379078 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:55:58.379078 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:55:58.379078 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:55:58.380260 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:55:58.380260 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:55:58.380260 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:55:58.380260 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:55:58.380260 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:55:58.380260 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:55:58.380260 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:55:58.380260 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:55:58.380260 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:55:58.380260 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:55:58.380260 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 16:55:58.380260 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:55:58.380260 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 581/1156] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 16:55:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:55:58.385668 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb0babe-0b87-4b51-9f65-dd2488876114 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:58.403382 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:58.483761 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb0babe-0b87-4b51-9f65-dd2488876114 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:58.516642 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff584fd1-d527-4b78-8d7a-c8425ec22f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:58.517238 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff584fd1-d527-4b78-8d7a-c8425ec22f40 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:58.517518 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff584fd1-d527-4b78-8d7a-c8425ec22f40 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:58.517787 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff584fd1-d527-4b78-8d7a-c8425ec22f40 None admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:58.518433 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff584fd1-d527-4b78-8d7a-c8425ec22f40 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:58.518731 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff584fd1-d527-4b78-8d7a-c8425ec22f40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:58.520124 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff584fd1-d527-4b78-8d7a-c8425ec22f40 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:58.528123 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 576/1157] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:55:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:58.533898 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e00626-9e85-43ae-ad3c-8081822c3aad None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:58.549163 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:58.631416 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e00626-9e85-43ae-ad3c-8081822c3aad None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:58.664703 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b1b4f6-8aa3-4a02-a975-534c202177eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:58.665311 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b1b4f6-8aa3-4a02-a975-534c202177eb None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:58.665590 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b1b4f6-8aa3-4a02-a975-534c202177eb None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:58.665860 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b1b4f6-8aa3-4a02-a975-534c202177eb None admin] PATH_INFO: `/v3/users/347296c34afd4772a2a8ec78cbc4437e` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:58.666983 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b1b4f6-8aa3-4a02-a975-534c202177eb None admin] RBAC: Authorizing `identity:get_user(user_id=347296c34afd4772a2a8ec78cbc4437e)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:58.668410 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b1b4f6-8aa3-4a02-a975-534c202177eb None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:58.669956 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 582/1158] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 16:55:58 2022] GET /identity/v3/users/347296c34afd4772a2a8ec78cbc4437e => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:55:58.675824 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff584fd1-d527-4b78-8d7a-c8425ec22f40 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:58.751953 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:58.772917 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff584fd1-d527-4b78-8d7a-c8425ec22f40 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:58.806884 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b6f8ed-769a-40d8-a4a8-a8866d11a5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:58.807515 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b6f8ed-769a-40d8-a4a8-a8866d11a5df None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:58.807798 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b6f8ed-769a-40d8-a4a8-a8866d11a5df None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:58.808075 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b6f8ed-769a-40d8-a4a8-a8866d11a5df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:58.873244 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b6f8ed-769a-40d8-a4a8-a8866d11a5df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:58.874699 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b6f8ed-769a-40d8-a4a8-a8866d11a5df None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:58.948204 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:58.969871 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 577/1159] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:55:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 16:55:58.976538 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b1b4f6-8aa3-4a02-a975-534c202177eb None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:59.054206 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:55:59.072869 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b1b4f6-8aa3-4a02-a975-534c202177eb None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:59.106487 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee00c031-892b-43a7-b4e3-0d7592bfc0e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:59.107136 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee00c031-892b-43a7-b4e3-0d7592bfc0e1 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:59.107421 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee00c031-892b-43a7-b4e3-0d7592bfc0e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:59.107694 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee00c031-892b-43a7-b4e3-0d7592bfc0e1 None admin] PATH_INFO: `/v3/projects/bdce85a7c68c4c9895fff4a356bce0df` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:59.108799 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee00c031-892b-43a7-b4e3-0d7592bfc0e1 None admin] RBAC: Authorizing `identity:get_project(project_id=bdce85a7c68c4c9895fff4a356bce0df)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:59.110229 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee00c031-892b-43a7-b4e3-0d7592bfc0e1 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:59.111889 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 583/1160] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 16:55:58 2022] GET /identity/v3/projects/bdce85a7c68c4c9895fff4a356bce0df => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:55:59.118028 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b6f8ed-769a-40d8-a4a8-a8866d11a5df None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:55:59.214127 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b6f8ed-769a-40d8-a4a8-a8866d11a5df None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:55:59.246516 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3d769a-8566-441f-810c-f514e75c6958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:55:59.247129 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3d769a-8566-441f-810c-f514e75c6958 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:55:59.247407 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3d769a-8566-441f-810c-f514e75c6958 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:55:59.247676 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3d769a-8566-441f-810c-f514e75c6958 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:55:59.248380 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3d769a-8566-441f-810c-f514e75c6958 None admin] RBAC: Adding query filter params (role.id=2c6966cfb10f4d72b0242e9fb5e13b17, scope.project.id=bdce85a7c68c4c9895fff4a356bce0df, user.id=347296c34afd4772a2a8ec78cbc4437e) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:55:59.248684 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3d769a-8566-441f-810c-f514e75c6958 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:55:59.250089 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3d769a-8566-441f-810c-f514e75c6958 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:55:59.255599 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 578/1161] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 16:55:59 2022] GET /identity/v3/role_assignments?user.id=347296c34afd4772a2a8ec78cbc4437e&scope.project.id=bdce85a7c68c4c9895fff4a356bce0df&role.id=2c6966cfb10f4d72b0242e9fb5e13b17 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:56:01.137449 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09927343-65dc-499d-9680-586f0b75b5ba None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:56:01.137996 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09927343-65dc-499d-9680-586f0b75b5ba None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:56:01.137996 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09927343-65dc-499d-9680-586f0b75b5ba None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:56:01.139392 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 584/1162] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:56:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:56:01.144571 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecbfed1-2bc2-415d-832e-c0d4a881a9b7 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:56:01.144956 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecbfed1-2bc2-415d-832e-c0d4a881a9b7 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:56:01.145127 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecbfed1-2bc2-415d-832e-c0d4a881a9b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:56:01.163794 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-3ecbfed1-2bc2-415d-832e-c0d4a881a9b7 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:56:01.164976 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-3ecbfed1-2bc2-415d-832e-c0d4a881a9b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:56:01.199215 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:56:01.230044 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 579/1163] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:56:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 16:56:01.498507 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09927343-65dc-499d-9680-586f0b75b5ba None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:56:01.597312 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09927343-65dc-499d-9680-586f0b75b5ba None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:56:01.629479 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad36de75-9757-4ada-9e7e-ca969934f5f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:56:01.630078 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36de75-9757-4ada-9e7e-ca969934f5f0 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:56:01.630360 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36de75-9757-4ada-9e7e-ca969934f5f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:56:01.631180 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36de75-9757-4ada-9e7e-ca969934f5f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:56:01.648470 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:56:01.698318 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad36de75-9757-4ada-9e7e-ca969934f5f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:56:01.699797 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad36de75-9757-4ada-9e7e-ca969934f5f0 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:56:01.793678 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 585/1164] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:56:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 16:56:01.799654 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecbfed1-2bc2-415d-832e-c0d4a881a9b7 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:56:01.894992 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecbfed1-2bc2-415d-832e-c0d4a881a9b7 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:56:01.927063 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dffddcd-0847-42cf-936f-c5d0f03f7a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:56:01.928355 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dffddcd-0847-42cf-936f-c5d0f03f7a1e None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:56:01.928756 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dffddcd-0847-42cf-936f-c5d0f03f7a1e None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:56:01.929031 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dffddcd-0847-42cf-936f-c5d0f03f7a1e None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:56:01.935363 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dffddcd-0847-42cf-936f-c5d0f03f7a1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:56:01.936800 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dffddcd-0847-42cf-936f-c5d0f03f7a1e None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:56:01.948603 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-2dffddcd-0847-42cf-936f-c5d0f03f7a1e None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 18 16:56:01.948603 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:56:01.948603 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 18 16:56:01.948603 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 18 16:56:01.948603 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 16:56:01.948603 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 16:56:01.948603 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 16:56:01.948603 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 16:56:01.948603 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 18 16:56:01.948603 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:56:01.948603 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 16:56:01.948603 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:56:01.948603 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:56:01.948603 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:56:01.948603 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:56:01.948603 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:56:01.948603 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:56:01.949754 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:56:01.949754 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:56:01.949754 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:56:01.949754 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:56:01.949754 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:56:01.949754 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:56:01.949754 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:56:01.949754 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:56:01.949754 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:56:01.949754 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:56:01.949754 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:56:01.949754 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:56:01.949754 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:56:01.949754 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:56:01.949754 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:56:01.949754 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:56:01.950775 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:56:01.950775 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:56:01.950775 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:56:01.950775 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:56:01.950775 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:56:01.950775 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:56:01.950775 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:56:01.950775 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:56:01.950775 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:56:01.950775 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:56:01.950775 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 18 16:56:01.950775 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:56:01.950775 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 18 16:56:01.950775 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:56:01.950775 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 580/1165] 10.222.0.82 () {56 vars in 1117 bytes} [Thu Aug 18 16:56:01 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:56:01.955544 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad36de75-9757-4ada-9e7e-ca969934f5f0 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:56:02.002758 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:56:02.054281 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad36de75-9757-4ada-9e7e-ca969934f5f0 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:56:02.086147 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef622a68-89c0-4a55-b99a-b7df93a48e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:56:02.086736 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef622a68-89c0-4a55-b99a-b7df93a48e24 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:56:02.087038 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef622a68-89c0-4a55-b99a-b7df93a48e24 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:56:02.087308 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef622a68-89c0-4a55-b99a-b7df93a48e24 None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:56:02.087928 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef622a68-89c0-4a55-b99a-b7df93a48e24 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:56:02.088223 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef622a68-89c0-4a55-b99a-b7df93a48e24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:56:02.089566 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef622a68-89c0-4a55-b99a-b7df93a48e24 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:56:02.096300 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 586/1166] 10.222.0.82 () {56 vars in 1115 bytes} [Thu Aug 18 16:56:01 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:56:03.984576 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42e07bf-f2d3-4468-b283-b4ed1f090615 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:56:03.985093 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42e07bf-f2d3-4468-b283-b4ed1f090615 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:56:03.985093 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42e07bf-f2d3-4468-b283-b4ed1f090615 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:56:03.986438 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 581/1167] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:56:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:56:03.991511 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea9f60-559d-4f6f-bf6b-5af6d50bb15f None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:56:03.991895 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea9f60-559d-4f6f-bf6b-5af6d50bb15f None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:56:03.992060 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea9f60-559d-4f6f-bf6b-5af6d50bb15f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:56:04.009515 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-82ea9f60-559d-4f6f-bf6b-5af6d50bb15f None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:56:04.010679 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-82ea9f60-559d-4f6f-bf6b-5af6d50bb15f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:56:04.075849 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 587/1168] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:56:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 16:56:04.343632 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42e07bf-f2d3-4468-b283-b4ed1f090615 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:56:04.374225 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:56:04.442442 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42e07bf-f2d3-4468-b283-b4ed1f090615 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:56:04.475920 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dda409-02ad-4f52-bc01-5e36fe1b451a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:56:04.476517 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dda409-02ad-4f52-bc01-5e36fe1b451a None admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:56:04.476798 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dda409-02ad-4f52-bc01-5e36fe1b451a None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:56:04.477068 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dda409-02ad-4f52-bc01-5e36fe1b451a None admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:56:04.477712 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dda409-02ad-4f52-bc01-5e36fe1b451a None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:56:04.479106 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dda409-02ad-4f52-bc01-5e36fe1b451a None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:56:04.552938 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 582/1169] 10.222.0.82 () {60 vars in 1138 bytes} [Thu Aug 18 16:56:04 2022] POST /identity/v3/domains => generated 267 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 16:56:06.475637 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0570ec-cbf6-4a97-9219-1de8d4f9ae92 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:56:06.476195 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0570ec-cbf6-4a97-9219-1de8d4f9ae92 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:56:06.476195 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0570ec-cbf6-4a97-9219-1de8d4f9ae92 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:56:06.477531 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 588/1170] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:56:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:56:06.482765 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6592cb08-8a44-47b4-a8a5-f93124acdd33 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:56:06.483168 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6592cb08-8a44-47b4-a8a5-f93124acdd33 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:56:06.483362 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6592cb08-8a44-47b4-a8a5-f93124acdd33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:56:06.502008 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-6592cb08-8a44-47b4-a8a5-f93124acdd33 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:56:06.503235 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-6592cb08-8a44-47b4-a8a5-f93124acdd33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:56:06.567572 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 583/1171] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:56:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 16:56:06.836218 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0570ec-cbf6-4a97-9219-1de8d4f9ae92 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:56:06.878359 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:56:06.934379 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0570ec-cbf6-4a97-9219-1de8d4f9ae92 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:56:06.967432 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72872780-fc2c-47e2-a3ff-042ef235aa7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:56:06.968048 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72872780-fc2c-47e2-a3ff-042ef235aa7d None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:56:06.968328 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72872780-fc2c-47e2-a3ff-042ef235aa7d None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:56:06.968600 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72872780-fc2c-47e2-a3ff-042ef235aa7d None admin] PATH_INFO: `/v3/domains/2531d0e85705456283df3ca8bf8c6243` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:56:06.978615 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72872780-fc2c-47e2-a3ff-042ef235aa7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=2531d0e85705456283df3ca8bf8c6243)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:56:06.980080 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72872780-fc2c-47e2-a3ff-042ef235aa7d None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:56:06.981628 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 589/1172] 10.222.0.82 () {56 vars in 1183 bytes} [Thu Aug 18 16:56:06 2022] GET /identity/v3/domains/2531d0e85705456283df3ca8bf8c6243 => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:56:06.987461 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6592cb08-8a44-47b4-a8a5-f93124acdd33 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:56:07.011810 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:56:07.083865 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6592cb08-8a44-47b4-a8a5-f93124acdd33 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:56:07.116086 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6086b978-8ee6-4ffd-81ce-53b7073caa14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:56:07.116683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6086b978-8ee6-4ffd-81ce-53b7073caa14 None admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:56:07.116958 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6086b978-8ee6-4ffd-81ce-53b7073caa14 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:56:07.117225 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6086b978-8ee6-4ffd-81ce-53b7073caa14 None admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:56:07.117854 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6086b978-8ee6-4ffd-81ce-53b7073caa14 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:56:07.119304 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6086b978-8ee6-4ffd-81ce-53b7073caa14 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:56:07.227327 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 584/1173] 10.222.0.82 () {60 vars in 1141 bytes} [Thu Aug 18 16:56:06 2022] POST /identity/v3/projects => generated 361 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 16:56:09.148272 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a574ab-2c1e-4feb-95dd-95d624651174 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:56:09.148790 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a574ab-2c1e-4feb-95dd-95d624651174 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:56:09.148880 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a574ab-2c1e-4feb-95dd-95d624651174 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:56:09.150152 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 590/1174] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:56:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:56:09.155372 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d22032-b217-4d31-aba5-a7c088a899ff None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:56:09.155758 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d22032-b217-4d31-aba5-a7c088a899ff None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:56:09.155920 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d22032-b217-4d31-aba5-a7c088a899ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:56:09.174262 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-91d22032-b217-4d31-aba5-a7c088a899ff None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:56:09.175493 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-91d22032-b217-4d31-aba5-a7c088a899ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:56:09.240257 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 585/1175] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:56:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 16:56:09.509555 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a574ab-2c1e-4feb-95dd-95d624651174 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:56:09.608078 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a574ab-2c1e-4feb-95dd-95d624651174 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:56:09.609527 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:56:09.640745 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed71affa-197d-45cf-9600-8a3534192183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:56:09.641357 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed71affa-197d-45cf-9600-8a3534192183 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:56:09.641631 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed71affa-197d-45cf-9600-8a3534192183 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:56:09.641899 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed71affa-197d-45cf-9600-8a3534192183 None admin] PATH_INFO: `/v3/domains/2531d0e85705456283df3ca8bf8c6243` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:56:09.642997 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed71affa-197d-45cf-9600-8a3534192183 None admin] RBAC: Authorizing `identity:get_domain(domain_id=2531d0e85705456283df3ca8bf8c6243)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:56:09.644411 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed71affa-197d-45cf-9600-8a3534192183 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:56:09.646919 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 591/1176] 10.222.0.82 () {56 vars in 1183 bytes} [Thu Aug 18 16:56:09 2022] GET /identity/v3/domains/2531d0e85705456283df3ca8bf8c6243 => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:56:09.652694 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d22032-b217-4d31-aba5-a7c088a899ff None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:56:09.655807 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:56:09.753299 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d22032-b217-4d31-aba5-a7c088a899ff None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:56:09.787165 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb84cba5-df6e-434e-a6f3-459b4a21bd8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:56:09.787773 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb84cba5-df6e-434e-a6f3-459b4a21bd8c None admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:56:09.788054 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb84cba5-df6e-434e-a6f3-459b4a21bd8c None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:56:09.788328 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb84cba5-df6e-434e-a6f3-459b4a21bd8c None admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:56:09.788958 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb84cba5-df6e-434e-a6f3-459b4a21bd8c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:56:09.790404 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb84cba5-df6e-434e-a6f3-459b4a21bd8c None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:56:09.874650 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 586/1177] 10.222.0.82 () {60 vars in 1132 bytes} [Thu Aug 18 16:56:09 2022] POST /identity/v3/users => generated 311 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 16:56:11.808476 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678dc6aa-1bbf-4e79-b641-c8c5a357fbb2 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:56:11.808950 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678dc6aa-1bbf-4e79-b641-c8c5a357fbb2 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:56:11.809046 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678dc6aa-1bbf-4e79-b641-c8c5a357fbb2 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:56:11.810354 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 592/1178] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:56:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:56:11.815480 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b5f4e3-8ab6-411e-85e9-9c667adbcb83 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:56:11.815865 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b5f4e3-8ab6-411e-85e9-9c667adbcb83 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:56:11.816039 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b5f4e3-8ab6-411e-85e9-9c667adbcb83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:56:11.834828 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-11b5f4e3-8ab6-411e-85e9-9c667adbcb83 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:56:11.836045 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-11b5f4e3-8ab6-411e-85e9-9c667adbcb83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:56:11.889793 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:56:11.901218 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 587/1179] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:56:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 16:56:12.202412 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678dc6aa-1bbf-4e79-b641-c8c5a357fbb2 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:56:12.302425 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678dc6aa-1bbf-4e79-b641-c8c5a357fbb2 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:56:12.336029 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed903f2-0762-4a5c-978c-f3a04826b39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:56:12.336643 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed903f2-0762-4a5c-978c-f3a04826b39d None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:56:12.336922 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed903f2-0762-4a5c-978c-f3a04826b39d None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:56:12.337192 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed903f2-0762-4a5c-978c-f3a04826b39d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:56:12.346908 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed903f2-0762-4a5c-978c-f3a04826b39d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:56:12.351625 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-2ed903f2-0762-4a5c-978c-f3a04826b39d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 16:56:12.351625 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:56:12.351625 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:56:12.351625 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:56:12.351625 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:56:12.351625 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:56:12.351625 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:56:12.351625 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:56:12.351625 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:56:12.351625 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:56:12.351625 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:56:12.351625 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:56:12.351625 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:56:12.351625 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:56:12.351625 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:56:12.351625 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:56:12.352771 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:56:12.352771 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:56:12.352771 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:56:12.352771 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:56:12.352771 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:56:12.352771 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:56:12.352771 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:56:12.352771 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:56:12.352771 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:56:12.352771 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:56:12.352771 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:56:12.352771 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:56:12.352771 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:56:12.352771 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:56:12.352771 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:56:12.352771 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:56:12.353969 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:56:12.353969 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:56:12.353969 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:56:12.353969 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:56:12.353969 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:56:12.353969 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:56:12.353969 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:56:12.353969 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:56:12.353969 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:56:12.353969 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:56:12.353969 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 16:56:12.353969 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:56:12.353969 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 593/1180] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 16:56:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:56:12.358690 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b5f4e3-8ab6-411e-85e9-9c667adbcb83 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:56:12.456325 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b5f4e3-8ab6-411e-85e9-9c667adbcb83 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:56:12.490598 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff87bc6-95db-4f52-a788-dd401b8ebf8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:56:12.491235 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff87bc6-95db-4f52-a788-dd401b8ebf8c None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:56:12.491523 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff87bc6-95db-4f52-a788-dd401b8ebf8c None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:56:12.491856 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff87bc6-95db-4f52-a788-dd401b8ebf8c None admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:56:12.492523 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff87bc6-95db-4f52-a788-dd401b8ebf8c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:56:12.492829 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff87bc6-95db-4f52-a788-dd401b8ebf8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:56:12.494220 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff87bc6-95db-4f52-a788-dd401b8ebf8c None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:56:12.502295 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 588/1181] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:56:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:56:12.509530 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed903f2-0762-4a5c-978c-f3a04826b39d None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:56:12.540537 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:56:12.608651 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed903f2-0762-4a5c-978c-f3a04826b39d None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:56:12.641671 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0aa535-636e-4b4e-97eb-36898dbe9c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:56:12.642284 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0aa535-636e-4b4e-97eb-36898dbe9c68 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:56:12.642563 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0aa535-636e-4b4e-97eb-36898dbe9c68 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:56:12.642835 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0aa535-636e-4b4e-97eb-36898dbe9c68 None admin] PATH_INFO: `/v3/users/e0e40973631a4d138bb5f308adbc3f94` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:56:12.655427 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0aa535-636e-4b4e-97eb-36898dbe9c68 None admin] RBAC: Authorizing `identity:get_user(user_id=e0e40973631a4d138bb5f308adbc3f94)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:56:12.656858 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0aa535-636e-4b4e-97eb-36898dbe9c68 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:56:12.658412 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 594/1182] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 16:56:12 2022] GET /identity/v3/users/e0e40973631a4d138bb5f308adbc3f94 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:56:12.664816 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff87bc6-95db-4f52-a788-dd401b8ebf8c None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:56:12.716800 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:56:12.763447 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff87bc6-95db-4f52-a788-dd401b8ebf8c None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:56:12.797231 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbcab2d-d2a5-4f96-93b9-20cfeb091b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:56:12.797832 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbcab2d-d2a5-4f96-93b9-20cfeb091b60 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:56:12.798135 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbcab2d-d2a5-4f96-93b9-20cfeb091b60 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:56:12.798391 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbcab2d-d2a5-4f96-93b9-20cfeb091b60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:56:12.863260 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbcab2d-d2a5-4f96-93b9-20cfeb091b60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:56:12.865511 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbcab2d-d2a5-4f96-93b9-20cfeb091b60 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:56:12.914318 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:56:12.960598 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 589/1183] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:56:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 16:56:12.966494 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0aa535-636e-4b4e-97eb-36898dbe9c68 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:56:13.051530 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:56:13.062874 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0aa535-636e-4b4e-97eb-36898dbe9c68 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:56:13.095755 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a699bae8-16f3-4347-85f1-089f3bcec129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:56:13.096375 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a699bae8-16f3-4347-85f1-089f3bcec129 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:56:13.096657 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a699bae8-16f3-4347-85f1-089f3bcec129 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:56:13.096930 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a699bae8-16f3-4347-85f1-089f3bcec129 None admin] PATH_INFO: `/v3/projects/23c04ce33f124f098488c5f221aeef37` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:56:13.098025 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a699bae8-16f3-4347-85f1-089f3bcec129 None admin] RBAC: Authorizing `identity:get_project(project_id=23c04ce33f124f098488c5f221aeef37)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:56:13.100213 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a699bae8-16f3-4347-85f1-089f3bcec129 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:56:13.102148 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 595/1184] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 16:56:12 2022] GET /identity/v3/projects/23c04ce33f124f098488c5f221aeef37 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:56:13.108173 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbcab2d-d2a5-4f96-93b9-20cfeb091b60 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:56:13.203751 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbcab2d-d2a5-4f96-93b9-20cfeb091b60 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:56:13.237893 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ed6b48-1ec5-4f5f-8d54-7cc023c8f53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:56:13.238491 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed6b48-1ec5-4f5f-8d54-7cc023c8f53c None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:56:13.238773 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed6b48-1ec5-4f5f-8d54-7cc023c8f53c None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:56:13.239165 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed6b48-1ec5-4f5f-8d54-7cc023c8f53c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:56:13.240143 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ed6b48-1ec5-4f5f-8d54-7cc023c8f53c None admin] RBAC: Adding query filter params (role.id=2c6966cfb10f4d72b0242e9fb5e13b17, scope.project.id=23c04ce33f124f098488c5f221aeef37, user.id=e0e40973631a4d138bb5f308adbc3f94) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:56:13.240459 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ed6b48-1ec5-4f5f-8d54-7cc023c8f53c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:56:13.241902 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ed6b48-1ec5-4f5f-8d54-7cc023c8f53c None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:56:13.247199 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 590/1185] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 16:56:13 2022] GET /identity/v3/role_assignments?user.id=e0e40973631a4d138bb5f308adbc3f94&scope.project.id=23c04ce33f124f098488c5f221aeef37&role.id=2c6966cfb10f4d72b0242e9fb5e13b17 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:56:15.114009 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb1d426-d7e0-434a-9fb1-28d2e59b4fee None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:56:15.114561 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb1d426-d7e0-434a-9fb1-28d2e59b4fee None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:56:15.114561 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb1d426-d7e0-434a-9fb1-28d2e59b4fee None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:56:15.115896 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 596/1186] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:56:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:56:15.121074 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42862009-2b08-4101-9351-cb45b524f759 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:56:15.121487 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42862009-2b08-4101-9351-cb45b524f759 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:56:15.121641 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42862009-2b08-4101-9351-cb45b524f759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:56:15.139767 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-42862009-2b08-4101-9351-cb45b524f759 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:56:15.140942 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-42862009-2b08-4101-9351-cb45b524f759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:56:15.151797 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:56:15.208595 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 591/1187] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:56:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 16:56:15.473141 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb1d426-d7e0-434a-9fb1-28d2e59b4fee None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:56:15.575064 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb1d426-d7e0-434a-9fb1-28d2e59b4fee None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:56:15.607991 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f45a24-dd3c-46a7-9e31-61670ac78b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:56:15.608606 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f45a24-dd3c-46a7-9e31-61670ac78b57 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:56:15.608885 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f45a24-dd3c-46a7-9e31-61670ac78b57 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:56:15.609157 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f45a24-dd3c-46a7-9e31-61670ac78b57 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:56:15.619730 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f45a24-dd3c-46a7-9e31-61670ac78b57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:56:15.624324 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-36f45a24-dd3c-46a7-9e31-61670ac78b57 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 16:56:15.624324 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:56:15.624324 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:56:15.624324 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:56:15.624324 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:56:15.624324 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:56:15.624324 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:56:15.624324 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:56:15.624324 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:56:15.624324 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:56:15.624324 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:56:15.624324 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:56:15.624324 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:56:15.624324 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:56:15.624324 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:56:15.624324 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:56:15.625370 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:56:15.625370 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:56:15.625370 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:56:15.625370 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:56:15.625370 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:56:15.625370 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:56:15.625370 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:56:15.625370 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:56:15.625370 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:56:15.625370 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:56:15.625370 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:56:15.625370 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:56:15.625370 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:56:15.625370 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:56:15.625370 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:56:15.625370 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:56:15.626368 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:56:15.626368 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:56:15.626368 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:56:15.626368 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:56:15.626368 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:56:15.626368 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:56:15.626368 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:56:15.626368 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:56:15.626368 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:56:15.626368 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:56:15.626368 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 16:56:15.626368 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:56:15.626368 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 597/1188] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 16:56:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:56:15.631649 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42862009-2b08-4101-9351-cb45b524f759 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:56:15.726004 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42862009-2b08-4101-9351-cb45b524f759 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:56:15.758090 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15366b86-ba05-4333-86ed-8e4809e3ad7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:56:15.758685 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15366b86-ba05-4333-86ed-8e4809e3ad7c None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:56:15.758988 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15366b86-ba05-4333-86ed-8e4809e3ad7c None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:56:15.759262 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15366b86-ba05-4333-86ed-8e4809e3ad7c None admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:56:15.759908 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15366b86-ba05-4333-86ed-8e4809e3ad7c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:56:15.760208 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15366b86-ba05-4333-86ed-8e4809e3ad7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:56:15.761571 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15366b86-ba05-4333-86ed-8e4809e3ad7c None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:56:15.769443 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 592/1189] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:56:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:56:15.775375 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f45a24-dd3c-46a7-9e31-61670ac78b57 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:56:15.794040 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:56:15.871845 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f45a24-dd3c-46a7-9e31-61670ac78b57 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:56:15.904144 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0286743b-1629-408a-9559-8a85f3b07a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:56:15.904753 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0286743b-1629-408a-9559-8a85f3b07a41 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:56:15.905030 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0286743b-1629-408a-9559-8a85f3b07a41 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:56:15.905295 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0286743b-1629-408a-9559-8a85f3b07a41 None admin] PATH_INFO: `/v3/users/e0e40973631a4d138bb5f308adbc3f94` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:56:15.906397 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0286743b-1629-408a-9559-8a85f3b07a41 None admin] RBAC: Authorizing `identity:get_user(user_id=e0e40973631a4d138bb5f308adbc3f94)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:56:15.907831 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0286743b-1629-408a-9559-8a85f3b07a41 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:56:15.909363 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 598/1190] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 16:56:15 2022] GET /identity/v3/users/e0e40973631a4d138bb5f308adbc3f94 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:56:15.915799 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15366b86-ba05-4333-86ed-8e4809e3ad7c None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:56:15.927603 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:56:16.015015 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15366b86-ba05-4333-86ed-8e4809e3ad7c None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:56:16.046692 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1457bbc3-5b8b-4c11-b4e2-994b5221796f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:56:16.047336 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1457bbc3-5b8b-4c11-b4e2-994b5221796f None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:56:16.047615 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1457bbc3-5b8b-4c11-b4e2-994b5221796f None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:56:16.047883 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1457bbc3-5b8b-4c11-b4e2-994b5221796f None admin] PATH_INFO: `/v3/projects/23c04ce33f124f098488c5f221aeef37` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:56:16.048957 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1457bbc3-5b8b-4c11-b4e2-994b5221796f None admin] RBAC: Authorizing `identity:get_project(project_id=23c04ce33f124f098488c5f221aeef37)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:56:16.050375 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1457bbc3-5b8b-4c11-b4e2-994b5221796f None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:56:16.052830 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 593/1191] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 16:56:15 2022] GET /identity/v3/projects/23c04ce33f124f098488c5f221aeef37 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:56:16.058643 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0286743b-1629-408a-9559-8a85f3b07a41 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:56:16.141412 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:56:16.156875 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0286743b-1629-408a-9559-8a85f3b07a41 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:56:16.190781 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4c12f4-c07e-4ca0-990b-db9c5bf9c10f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:56:16.191456 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c12f4-c07e-4ca0-990b-db9c5bf9c10f None admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:56:16.191773 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c12f4-c07e-4ca0-990b-db9c5bf9c10f None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:56:16.192219 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c12f4-c07e-4ca0-990b-db9c5bf9c10f None admin] PATH_INFO: `/v3/projects/23c04ce33f124f098488c5f221aeef37/users/e0e40973631a4d138bb5f308adbc3f94/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:56:16.194121 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4c12f4-c07e-4ca0-990b-db9c5bf9c10f None admin] RBAC: Authorizing `identity:create_grant(project_id=23c04ce33f124f098488c5f221aeef37, user_id=e0e40973631a4d138bb5f308adbc3f94, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:56:16.195576 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4c12f4-c07e-4ca0-990b-db9c5bf9c10f None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:56:16.268675 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 599/1192] 10.222.0.82 () {58 vars in 1439 bytes} [Thu Aug 18 16:56:16 2022] PUT /identity/v3/projects/23c04ce33f124f098488c5f221aeef37/users/e0e40973631a4d138bb5f308adbc3f94/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 16:56:18.142303 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b057354-0510-4726-81b5-e84998d07f40 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:56:18.142848 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b057354-0510-4726-81b5-e84998d07f40 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:56:18.142988 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b057354-0510-4726-81b5-e84998d07f40 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:56:18.144245 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 594/1193] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:56:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:56:18.150362 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f4b8d-7cab-4f1e-8066-dd41c45ce1a0 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:56:18.150783 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f4b8d-7cab-4f1e-8066-dd41c45ce1a0 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:56:18.150943 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f4b8d-7cab-4f1e-8066-dd41c45ce1a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:56:18.168292 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-b54f4b8d-7cab-4f1e-8066-dd41c45ce1a0 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:56:18.169451 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-b54f4b8d-7cab-4f1e-8066-dd41c45ce1a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:56:18.235765 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 600/1194] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:56:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 16:56:18.510077 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b057354-0510-4726-81b5-e84998d07f40 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:56:18.590094 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:56:18.613721 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b057354-0510-4726-81b5-e84998d07f40 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:56:18.648641 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab37fdd-3127-4ce5-8dc4-3f81003fd067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:56:18.649269 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab37fdd-3127-4ce5-8dc4-3f81003fd067 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:56:18.649549 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab37fdd-3127-4ce5-8dc4-3f81003fd067 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:56:18.650072 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab37fdd-3127-4ce5-8dc4-3f81003fd067 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:56:18.661028 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab37fdd-3127-4ce5-8dc4-3f81003fd067 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:56:18.665688 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-aab37fdd-3127-4ce5-8dc4-3f81003fd067 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 16:56:18.665688 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:56:18.665688 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:56:18.665688 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:56:18.665688 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:56:18.665688 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:56:18.665688 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:56:18.665688 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:56:18.665688 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:56:18.665688 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:56:18.665688 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:56:18.665688 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:56:18.665688 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:56:18.665688 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:56:18.665688 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:56:18.665688 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:56:18.666811 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:56:18.666811 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:56:18.666811 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:56:18.666811 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:56:18.666811 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:56:18.666811 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:56:18.666811 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:56:18.666811 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:56:18.666811 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:56:18.666811 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:56:18.666811 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:56:18.666811 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:56:18.666811 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:56:18.666811 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:56:18.666811 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:56:18.666811 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:56:18.667862 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:56:18.667862 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:56:18.667862 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:56:18.667862 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:56:18.667862 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:56:18.667862 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:56:18.667862 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:56:18.667862 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:56:18.667862 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:56:18.667862 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:56:18.667862 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 16:56:18.667862 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:56:18.667862 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 595/1195] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 16:56:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:56:18.673281 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54f4b8d-7cab-4f1e-8066-dd41c45ce1a0 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:56:18.750848 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:56:18.776904 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54f4b8d-7cab-4f1e-8066-dd41c45ce1a0 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:56:18.820260 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83add884-2cee-44e6-be28-b9b4266c6903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:56:18.820867 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83add884-2cee-44e6-be28-b9b4266c6903 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:56:18.821142 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83add884-2cee-44e6-be28-b9b4266c6903 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:56:18.821411 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83add884-2cee-44e6-be28-b9b4266c6903 None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:56:18.822062 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83add884-2cee-44e6-be28-b9b4266c6903 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:56:18.822359 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83add884-2cee-44e6-be28-b9b4266c6903 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:56:18.823750 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83add884-2cee-44e6-be28-b9b4266c6903 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:56:18.831819 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 601/1196] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:56:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:56:18.838536 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab37fdd-3127-4ce5-8dc4-3f81003fd067 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:56:18.938194 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab37fdd-3127-4ce5-8dc4-3f81003fd067 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:56:18.972583 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d64afe9-3893-44ae-9b2b-880faf89dc5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:56:18.973357 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d64afe9-3893-44ae-9b2b-880faf89dc5e None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:56:18.973640 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d64afe9-3893-44ae-9b2b-880faf89dc5e None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:56:18.973915 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d64afe9-3893-44ae-9b2b-880faf89dc5e None admin] PATH_INFO: `/v3/users/e0e40973631a4d138bb5f308adbc3f94` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:56:18.975959 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d64afe9-3893-44ae-9b2b-880faf89dc5e None admin] RBAC: Authorizing `identity:get_user(user_id=e0e40973631a4d138bb5f308adbc3f94)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:56:18.976530 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d64afe9-3893-44ae-9b2b-880faf89dc5e None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:56:18.978104 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 596/1197] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 16:56:18 2022] GET /identity/v3/users/e0e40973631a4d138bb5f308adbc3f94 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:56:18.984775 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83add884-2cee-44e6-be28-b9b4266c6903 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:56:19.096491 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83add884-2cee-44e6-be28-b9b4266c6903 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:56:19.128533 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea81ea2-1e40-491a-bd75-fa5d682e51ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:56:19.129825 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea81ea2-1e40-491a-bd75-fa5d682e51ce None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:56:19.130107 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea81ea2-1e40-491a-bd75-fa5d682e51ce None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:56:19.130378 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea81ea2-1e40-491a-bd75-fa5d682e51ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:56:19.133711 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:56:19.195552 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea81ea2-1e40-491a-bd75-fa5d682e51ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:56:19.197019 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea81ea2-1e40-491a-bd75-fa5d682e51ce None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:56:19.298088 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 602/1198] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:56:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 16:56:19.304935 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d64afe9-3893-44ae-9b2b-880faf89dc5e None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:56:19.308751 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:56:19.402618 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d64afe9-3893-44ae-9b2b-880faf89dc5e None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:56:19.436358 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197b9534-8d62-4c48-89f6-e14bb02c7597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:56:19.436981 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197b9534-8d62-4c48-89f6-e14bb02c7597 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:56:19.437262 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197b9534-8d62-4c48-89f6-e14bb02c7597 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:56:19.437536 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197b9534-8d62-4c48-89f6-e14bb02c7597 None admin] PATH_INFO: `/v3/projects/23c04ce33f124f098488c5f221aeef37` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:56:19.438626 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197b9534-8d62-4c48-89f6-e14bb02c7597 None admin] RBAC: Authorizing `identity:get_project(project_id=23c04ce33f124f098488c5f221aeef37)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:56:19.440104 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197b9534-8d62-4c48-89f6-e14bb02c7597 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:56:19.441960 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 597/1199] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 16:56:19 2022] GET /identity/v3/projects/23c04ce33f124f098488c5f221aeef37 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:56:19.448316 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea81ea2-1e40-491a-bd75-fa5d682e51ce None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:56:19.481911 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:56:19.545261 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea81ea2-1e40-491a-bd75-fa5d682e51ce None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:56:19.579283 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dbd3a0-cf6c-4937-b9fc-d0de6f4d9841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:56:19.579876 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dbd3a0-cf6c-4937-b9fc-d0de6f4d9841 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:56:19.580159 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dbd3a0-cf6c-4937-b9fc-d0de6f4d9841 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:56:19.580433 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dbd3a0-cf6c-4937-b9fc-d0de6f4d9841 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:56:19.581226 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dbd3a0-cf6c-4937-b9fc-d0de6f4d9841 None admin] RBAC: Adding query filter params (role.id=2c6966cfb10f4d72b0242e9fb5e13b17, scope.project.id=23c04ce33f124f098488c5f221aeef37, user.id=e0e40973631a4d138bb5f308adbc3f94) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:56:19.581539 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dbd3a0-cf6c-4937-b9fc-d0de6f4d9841 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:56:19.583411 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dbd3a0-cf6c-4937-b9fc-d0de6f4d9841 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:56:19.589032 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 603/1200] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 16:56:19 2022] GET /identity/v3/role_assignments?user.id=e0e40973631a4d138bb5f308adbc3f94&scope.project.id=23c04ce33f124f098488c5f221aeef37&role.id=2c6966cfb10f4d72b0242e9fb5e13b17 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:56:29.432178 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=64 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:56:29.586410 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=50 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:57:31.775687 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffce13a-3b01-42ab-8d65-1a7b5c22ad13 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:31.776196 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffce13a-3b01-42ab-8d65-1a7b5c22ad13 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:31.776298 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffce13a-3b01-42ab-8d65-1a7b5c22ad13 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:31.777617 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 598/1201] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:57:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:57:31.782871 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9369acbe-7d26-410a-ad52-019c267c2aa0 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:31.783338 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9369acbe-7d26-410a-ad52-019c267c2aa0 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:31.783436 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9369acbe-7d26-410a-ad52-019c267c2aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:31.802321 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-9369acbe-7d26-410a-ad52-019c267c2aa0 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:57:31.805681 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-9369acbe-7d26-410a-ad52-019c267c2aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:57:31.868922 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 604/1202] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:57:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 16:57:32.142749 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffce13a-3b01-42ab-8d65-1a7b5c22ad13 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:32.243574 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffce13a-3b01-42ab-8d65-1a7b5c22ad13 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:32.276344 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6629475f-1de6-4cb9-8e07-d0c271f60ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:32.276968 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629475f-1de6-4cb9-8e07-d0c271f60ac3 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:32.277249 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629475f-1de6-4cb9-8e07-d0c271f60ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:32.277525 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629475f-1de6-4cb9-8e07-d0c271f60ac3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:32.289717 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6629475f-1de6-4cb9-8e07-d0c271f60ac3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:32.291667 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6629475f-1de6-4cb9-8e07-d0c271f60ac3 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:32.306187 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-6629475f-1de6-4cb9-8e07-d0c271f60ac3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:57:32.306187 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:57:32.306187 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:57:32.306187 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:57:32.306187 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:57:32.306187 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:57:32.306187 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:57:32.306187 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:57:32.306187 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:57:32.306187 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:57:32.306187 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:57:32.306187 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:57:32.306187 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:57:32.306187 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:57:32.306187 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:57:32.306187 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:57:32.307590 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:57:32.307590 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:57:32.307590 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:57:32.307590 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:57:32.307590 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:57:32.307590 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:57:32.307590 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:57:32.307590 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:57:32.307590 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:57:32.307590 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:57:32.307590 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:57:32.307590 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:57:32.307590 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:57:32.307590 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:57:32.307590 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:57:32.307590 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:57:32.307590 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:57:32.308846 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:57:32.308846 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:57:32.308846 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:57:32.308846 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 599/1203] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:57:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:57:32.313329 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9369acbe-7d26-410a-ad52-019c267c2aa0 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:32.410100 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9369acbe-7d26-410a-ad52-019c267c2aa0 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:32.437628 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:57:32.443785 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d13a4f5-a18a-49d2-ac0f-a6a40b73922d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:32.446608 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d13a4f5-a18a-49d2-ac0f-a6a40b73922d None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:32.446608 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d13a4f5-a18a-49d2-ac0f-a6a40b73922d None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:32.446608 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d13a4f5-a18a-49d2-ac0f-a6a40b73922d None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:32.446608 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d13a4f5-a18a-49d2-ac0f-a6a40b73922d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:57:32.446608 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d13a4f5-a18a-49d2-ac0f-a6a40b73922d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:32.447186 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d13a4f5-a18a-49d2-ac0f-a6a40b73922d None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:32.458239 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 605/1204] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:57:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:57:32.464133 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6629475f-1de6-4cb9-8e07-d0c271f60ac3 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:32.517209 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:57:32.565002 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6629475f-1de6-4cb9-8e07-d0c271f60ac3 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:32.599167 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af5d39a-f83a-4c5f-b3f4-12091ba379cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:32.599771 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5d39a-f83a-4c5f-b3f4-12091ba379cc None admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:32.600046 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5d39a-f83a-4c5f-b3f4-12091ba379cc None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:32.600349 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5d39a-f83a-4c5f-b3f4-12091ba379cc None admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:32.601020 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af5d39a-f83a-4c5f-b3f4-12091ba379cc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:32.602770 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af5d39a-f83a-4c5f-b3f4-12091ba379cc None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:32.688287 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 600/1205] 10.222.0.82 () {60 vars in 1132 bytes} [Thu Aug 18 16:57:32 2022] POST /identity/v3/users => generated 251 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 16:57:34.614831 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb7a254-61bc-49de-b01a-5666a1841ded None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:34.615430 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb7a254-61bc-49de-b01a-5666a1841ded None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:34.615430 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb7a254-61bc-49de-b01a-5666a1841ded None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:34.616824 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 606/1206] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:57:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:57:34.621890 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d2bc25-a0a5-4f3b-a9a8-0e56e33c1a1b None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:34.622331 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d2bc25-a0a5-4f3b-a9a8-0e56e33c1a1b None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:34.622431 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d2bc25-a0a5-4f3b-a9a8-0e56e33c1a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:34.640110 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-81d2bc25-a0a5-4f3b-a9a8-0e56e33c1a1b None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:57:34.641278 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-81d2bc25-a0a5-4f3b-a9a8-0e56e33c1a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:57:34.710535 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 601/1207] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:57:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 16:57:34.982055 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb7a254-61bc-49de-b01a-5666a1841ded None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:35.082720 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb7a254-61bc-49de-b01a-5666a1841ded None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:35.115760 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9364731-90d6-459f-8e84-fd115564341a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:35.116374 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9364731-90d6-459f-8e84-fd115564341a None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:35.116652 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9364731-90d6-459f-8e84-fd115564341a None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:35.116921 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9364731-90d6-459f-8e84-fd115564341a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:35.126482 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9364731-90d6-459f-8e84-fd115564341a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:57:35.132033 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-e9364731-90d6-459f-8e84-fd115564341a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 16:57:35.132033 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:57:35.132033 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:57:35.132033 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:57:35.132033 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:57:35.132033 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:57:35.132033 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:57:35.132033 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:57:35.132033 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:57:35.132033 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:57:35.132033 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:57:35.132033 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:57:35.132033 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:57:35.132033 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:57:35.132033 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:57:35.132033 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:57:35.133220 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:57:35.133220 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:57:35.133220 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:57:35.133220 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:57:35.133220 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:57:35.133220 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:57:35.133220 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:57:35.133220 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:57:35.133220 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:57:35.133220 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:57:35.133220 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:57:35.133220 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:57:35.133220 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:57:35.133220 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:57:35.133220 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:57:35.133220 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:57:35.134528 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:57:35.134528 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:57:35.134528 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:57:35.134528 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:57:35.134528 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:57:35.134528 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:57:35.134528 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:57:35.134528 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:57:35.134528 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:57:35.134528 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:57:35.134528 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 16:57:35.134528 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:57:35.134528 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 607/1208] 10.222.0.82 () {56 vars in 1102 bytes} [Thu Aug 18 16:57:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:57:35.139204 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d2bc25-a0a5-4f3b-a9a8-0e56e33c1a1b None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:35.178773 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:57:35.236527 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d2bc25-a0a5-4f3b-a9a8-0e56e33c1a1b None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:35.270638 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea014c5-521f-4c3b-81fc-d920c6c79e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:35.271270 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea014c5-521f-4c3b-81fc-d920c6c79e09 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:35.271554 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea014c5-521f-4c3b-81fc-d920c6c79e09 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:35.271830 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea014c5-521f-4c3b-81fc-d920c6c79e09 None admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:35.272523 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea014c5-521f-4c3b-81fc-d920c6c79e09 None admin] RBAC: Adding query filter params (name=service) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:57:35.272815 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea014c5-521f-4c3b-81fc-d920c6c79e09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:35.274195 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea014c5-521f-4c3b-81fc-d920c6c79e09 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:35.281934 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 602/1209] 10.222.0.82 () {56 vars in 1103 bytes} [Thu Aug 18 16:57:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:57:35.288074 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9364731-90d6-459f-8e84-fd115564341a None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:35.338034 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:57:35.385909 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9364731-90d6-459f-8e84-fd115564341a None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:35.418515 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396bb824-b70a-427d-b0b0-d46b5443dd5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:35.419227 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396bb824-b70a-427d-b0b0-d46b5443dd5f None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:35.422862 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396bb824-b70a-427d-b0b0-d46b5443dd5f None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:35.422862 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396bb824-b70a-427d-b0b0-d46b5443dd5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:35.430160 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396bb824-b70a-427d-b0b0-d46b5443dd5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:35.431630 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396bb824-b70a-427d-b0b0-d46b5443dd5f None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:35.446885 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-396bb824-b70a-427d-b0b0-d46b5443dd5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:57:35.446885 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:57:35.446885 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:57:35.446885 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:57:35.446885 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:57:35.446885 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:57:35.446885 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:57:35.446885 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:57:35.446885 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:57:35.446885 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:57:35.446885 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:57:35.446885 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:57:35.446885 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:57:35.446885 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:57:35.446885 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:57:35.446885 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:57:35.448012 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:57:35.448012 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:57:35.448012 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:57:35.448012 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:57:35.448012 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:57:35.448012 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:57:35.448012 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:57:35.448012 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:57:35.448012 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:57:35.448012 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:57:35.448012 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:57:35.448012 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:57:35.448012 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:57:35.448012 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:57:35.448012 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:57:35.448012 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:57:35.448012 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:57:35.449083 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:57:35.449083 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:57:35.449083 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:57:35.449083 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 608/1210] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:57:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:57:35.454343 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea014c5-521f-4c3b-81fc-d920c6c79e09 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:35.550587 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:57:35.554072 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea014c5-521f-4c3b-81fc-d920c6c79e09 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:35.586908 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec49ff42-228b-445f-84c6-50cd599d8272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:35.587623 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec49ff42-228b-445f-84c6-50cd599d8272 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:35.587902 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec49ff42-228b-445f-84c6-50cd599d8272 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:35.588176 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec49ff42-228b-445f-84c6-50cd599d8272 None admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:35.588834 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec49ff42-228b-445f-84c6-50cd599d8272 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:57:35.589139 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec49ff42-228b-445f-84c6-50cd599d8272 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:35.590546 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec49ff42-228b-445f-84c6-50cd599d8272 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:35.602418 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 603/1211] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:57:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:57:35.608457 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396bb824-b70a-427d-b0b0-d46b5443dd5f None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:35.708349 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396bb824-b70a-427d-b0b0-d46b5443dd5f None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:35.709845 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:57:35.741690 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bd6f65-6d4c-44d0-8009-ea2d5c86419c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:35.742307 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bd6f65-6d4c-44d0-8009-ea2d5c86419c None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:35.742581 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bd6f65-6d4c-44d0-8009-ea2d5c86419c None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:35.742846 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bd6f65-6d4c-44d0-8009-ea2d5c86419c None admin] PATH_INFO: `/v3/users/placement` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:35.749455 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bd6f65-6d4c-44d0-8009-ea2d5c86419c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:35.750884 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bd6f65-6d4c-44d0-8009-ea2d5c86419c None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:35.763068 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-d3bd6f65-6d4c-44d0-8009-ea2d5c86419c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 18 16:57:35.763068 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:57:35.763068 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:57:35.763068 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:57:35.763068 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:57:35.763068 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:57:35.763068 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:57:35.763068 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:57:35.763068 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:57:35.763068 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:57:35.763068 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:57:35.763068 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:57:35.763068 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 16:57:35.763068 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 16:57:35.763068 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 16:57:35.763068 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 16:57:35.766722 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:57:35.766722 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:57:35.766722 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 16:57:35.766722 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:57:35.766722 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 16:57:35.766722 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:57:35.766722 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:57:35.766722 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:57:35.766722 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:57:35.766722 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:57:35.766722 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:57:35.766722 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:57:35.766722 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:57:35.766722 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:57:35.766722 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:57:35.766722 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:57:35.766722 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:57:35.766722 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:57:35.768440 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:57:35.768440 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:57:35.768440 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 16:57:35.768440 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 16:57:35.768440 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 16:57:35.768440 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 16:57:35.768440 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 16:57:35.768440 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 16:57:35.768440 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 18 16:57:35.768440 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:57:35.768440 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 609/1212] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:57:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:57:35.771763 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec49ff42-228b-445f-84c6-50cd599d8272 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:35.868459 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec49ff42-228b-445f-84c6-50cd599d8272 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:35.902089 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25285875-e664-4cbb-baa5-4b4478261f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:35.902690 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25285875-e664-4cbb-baa5-4b4478261f85 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:35.902991 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25285875-e664-4cbb-baa5-4b4478261f85 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:35.903269 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25285875-e664-4cbb-baa5-4b4478261f85 None admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:35.903979 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25285875-e664-4cbb-baa5-4b4478261f85 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:57:35.904284 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25285875-e664-4cbb-baa5-4b4478261f85 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:35.905714 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25285875-e664-4cbb-baa5-4b4478261f85 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:35.922625 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 604/1213] 10.222.0.82 () {56 vars in 1143 bytes} [Thu Aug 18 16:57:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:57:35.928960 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bd6f65-6d4c-44d0-8009-ea2d5c86419c None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:36.026363 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bd6f65-6d4c-44d0-8009-ea2d5c86419c None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:36.060944 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234a757d-4d44-465b-9665-505e85cf00cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:36.061685 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a757d-4d44-465b-9665-505e85cf00cf None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:36.061984 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a757d-4d44-465b-9665-505e85cf00cf None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:36.062259 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a757d-4d44-465b-9665-505e85cf00cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:36.092832 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:57:36.132069 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234a757d-4d44-465b-9665-505e85cf00cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:36.133450 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234a757d-4d44-465b-9665-505e85cf00cf None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:36.227786 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 610/1214] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:57:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 16:57:36.233718 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25285875-e664-4cbb-baa5-4b4478261f85 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:36.252787 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:57:36.334427 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25285875-e664-4cbb-baa5-4b4478261f85 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:36.368975 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b895ae-810d-4ba7-a973-63d4b9b97935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:36.369619 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b895ae-810d-4ba7-a973-63d4b9b97935 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:36.369923 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b895ae-810d-4ba7-a973-63d4b9b97935 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:36.370208 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b895ae-810d-4ba7-a973-63d4b9b97935 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:36.380616 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b895ae-810d-4ba7-a973-63d4b9b97935 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:36.382142 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b895ae-810d-4ba7-a973-63d4b9b97935 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:36.397149 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-59b895ae-810d-4ba7-a973-63d4b9b97935 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:57:36.397149 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:57:36.397149 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:57:36.397149 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:57:36.397149 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:57:36.397149 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:57:36.397149 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:57:36.397149 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:57:36.397149 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:57:36.397149 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:57:36.397149 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:57:36.397149 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:57:36.397149 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:57:36.397149 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:57:36.397149 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:57:36.397149 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:57:36.398180 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:57:36.398180 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:57:36.398180 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:57:36.398180 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:57:36.398180 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:57:36.398180 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:57:36.398180 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:57:36.398180 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:57:36.398180 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:57:36.398180 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:57:36.398180 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:57:36.398180 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:57:36.398180 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:57:36.398180 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:57:36.398180 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:57:36.398180 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:57:36.398180 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:57:36.399165 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:57:36.399165 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:57:36.399165 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:57:36.399165 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 605/1215] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:57:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:57:36.404054 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234a757d-4d44-465b-9665-505e85cf00cf None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:36.465349 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:57:36.501968 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234a757d-4d44-465b-9665-505e85cf00cf None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:36.534976 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0438f8b5-d108-4f4d-9afb-17e4cef9aa60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:36.535597 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0438f8b5-d108-4f4d-9afb-17e4cef9aa60 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:36.536001 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0438f8b5-d108-4f4d-9afb-17e4cef9aa60 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:36.536279 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0438f8b5-d108-4f4d-9afb-17e4cef9aa60 None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:36.536914 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0438f8b5-d108-4f4d-9afb-17e4cef9aa60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:57:36.537219 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0438f8b5-d108-4f4d-9afb-17e4cef9aa60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:36.538619 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0438f8b5-d108-4f4d-9afb-17e4cef9aa60 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:36.549930 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 611/1216] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:57:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:57:36.558643 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b895ae-810d-4ba7-a973-63d4b9b97935 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:36.625677 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:57:36.655078 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b895ae-810d-4ba7-a973-63d4b9b97935 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:36.687584 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db2cfbb-aba6-497a-867b-761a5f496d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:36.688202 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db2cfbb-aba6-497a-867b-761a5f496d35 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:36.688483 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db2cfbb-aba6-497a-867b-761a5f496d35 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:36.688756 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db2cfbb-aba6-497a-867b-761a5f496d35 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:36.695486 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db2cfbb-aba6-497a-867b-761a5f496d35 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:36.696936 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db2cfbb-aba6-497a-867b-761a5f496d35 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:36.708711 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-7db2cfbb-aba6-497a-867b-761a5f496d35 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:57:36.708711 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:57:36.708711 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:57:36.708711 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:57:36.708711 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:57:36.708711 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:57:36.708711 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:57:36.708711 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:57:36.708711 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:57:36.708711 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:57:36.708711 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:57:36.708711 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:57:36.708711 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 16:57:36.708711 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 16:57:36.708711 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 16:57:36.709669 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 16:57:36.709669 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:57:36.709669 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:57:36.709669 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:57:36.709669 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:57:36.709669 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:57:36.709669 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:57:36.709669 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:57:36.709669 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:57:36.709669 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:57:36.709669 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:57:36.709669 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:57:36.709669 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:57:36.709669 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:57:36.709669 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:57:36.709669 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:57:36.709669 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:57:36.710775 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 16:57:36.710775 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 16:57:36.710775 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 16:57:36.710775 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 16:57:36.710775 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 16:57:36.710775 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 16:57:36.710775 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:57:36.710775 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:57:36.710775 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 606/1217] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 16:57:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:57:36.716669 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0438f8b5-d108-4f4d-9afb-17e4cef9aa60 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:36.815151 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0438f8b5-d108-4f4d-9afb-17e4cef9aa60 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:36.837683 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:57:36.849115 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ec062f-6088-4f1a-9ac5-d3c74b7149d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:36.849752 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ec062f-6088-4f1a-9ac5-d3c74b7149d7 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:36.850025 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ec062f-6088-4f1a-9ac5-d3c74b7149d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:36.850293 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ec062f-6088-4f1a-9ac5-d3c74b7149d7 None admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:36.851092 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ec062f-6088-4f1a-9ac5-d3c74b7149d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:57:36.851401 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ec062f-6088-4f1a-9ac5-d3c74b7149d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:36.852815 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ec062f-6088-4f1a-9ac5-d3c74b7149d7 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:36.864037 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 612/1218] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 16:57:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:57:36.870312 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db2cfbb-aba6-497a-867b-761a5f496d35 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:36.971295 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db2cfbb-aba6-497a-867b-761a5f496d35 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:36.998304 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:57:37.003827 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596fa1f0-15d4-4bd1-8add-59cf961aecd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:37.004464 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596fa1f0-15d4-4bd1-8add-59cf961aecd5 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:37.004857 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596fa1f0-15d4-4bd1-8add-59cf961aecd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:37.005208 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596fa1f0-15d4-4bd1-8add-59cf961aecd5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:37.005924 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596fa1f0-15d4-4bd1-8add-59cf961aecd5 None admin] RBAC: Adding query filter params (role.id=5e0f402e1af84b44a723ee0730315d74, scope.project.id=2293c0aab61f44849d0d48b214cde7ea, user.id=29653ba1a2a74232972ef7a9000518b0) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:57:37.006235 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596fa1f0-15d4-4bd1-8add-59cf961aecd5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:37.007703 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596fa1f0-15d4-4bd1-8add-59cf961aecd5 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:37.013331 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 607/1219] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 16:57:36 2022] GET /identity/v3/role_assignments?user.id=29653ba1a2a74232972ef7a9000518b0&scope.project.id=2293c0aab61f44849d0d48b214cde7ea&role.id=5e0f402e1af84b44a723ee0730315d74 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:57:38.882871 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049de1f1-04fe-46e3-8f5f-f1ee0526d248 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:38.883436 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049de1f1-04fe-46e3-8f5f-f1ee0526d248 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:38.883436 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049de1f1-04fe-46e3-8f5f-f1ee0526d248 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:38.884770 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 613/1220] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:57:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:57:38.890032 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ef421-776b-4c37-b4f6-382f55ab240f None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:38.890422 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ef421-776b-4c37-b4f6-382f55ab240f None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:38.890584 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ef421-776b-4c37-b4f6-382f55ab240f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:38.909039 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-893ef421-776b-4c37-b4f6-382f55ab240f None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:57:38.910279 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-893ef421-776b-4c37-b4f6-382f55ab240f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:57:38.976937 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 608/1221] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:57:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 16:57:39.248746 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049de1f1-04fe-46e3-8f5f-f1ee0526d248 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:39.351126 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049de1f1-04fe-46e3-8f5f-f1ee0526d248 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:39.385247 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783a0f55-fe4f-4fbd-b2ca-0ce03fe010a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:39.385865 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783a0f55-fe4f-4fbd-b2ca-0ce03fe010a7 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:39.386144 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783a0f55-fe4f-4fbd-b2ca-0ce03fe010a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:39.386411 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783a0f55-fe4f-4fbd-b2ca-0ce03fe010a7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:39.396209 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783a0f55-fe4f-4fbd-b2ca-0ce03fe010a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:57:39.400873 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-783a0f55-fe4f-4fbd-b2ca-0ce03fe010a7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 16:57:39.400873 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:57:39.400873 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:57:39.400873 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:57:39.400873 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:57:39.400873 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:57:39.400873 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:57:39.400873 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:57:39.400873 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:57:39.400873 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:57:39.400873 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:57:39.400873 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:57:39.400873 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:57:39.400873 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:57:39.400873 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:57:39.400873 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:57:39.401863 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:57:39.401863 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:57:39.401863 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:57:39.401863 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:57:39.401863 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:57:39.401863 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:57:39.401863 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:57:39.401863 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:57:39.401863 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:57:39.401863 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:57:39.401863 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:57:39.401863 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:57:39.401863 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:57:39.401863 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:57:39.401863 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:57:39.401863 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:57:39.402744 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:57:39.402744 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:57:39.402744 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:57:39.402744 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:57:39.402744 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:57:39.402744 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:57:39.402744 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:57:39.402744 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:57:39.402744 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:57:39.402744 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:57:39.402744 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 16:57:39.402744 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:57:39.402744 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 614/1222] 10.222.0.82 () {56 vars in 1102 bytes} [Thu Aug 18 16:57:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 16:57:39.408042 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893ef421-776b-4c37-b4f6-382f55ab240f None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:39.506846 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893ef421-776b-4c37-b4f6-382f55ab240f None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:39.529279 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:57:39.540233 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2555a1c-c3d3-4f6d-9c8d-8de2f035282a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:39.540879 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2555a1c-c3d3-4f6d-9c8d-8de2f035282a None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:39.541192 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2555a1c-c3d3-4f6d-9c8d-8de2f035282a None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:39.541461 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2555a1c-c3d3-4f6d-9c8d-8de2f035282a None admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:39.542113 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2555a1c-c3d3-4f6d-9c8d-8de2f035282a None admin] RBAC: Adding query filter params (name=service) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:57:39.542412 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2555a1c-c3d3-4f6d-9c8d-8de2f035282a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:39.543817 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2555a1c-c3d3-4f6d-9c8d-8de2f035282a None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:39.552357 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 609/1223] 10.222.0.82 () {56 vars in 1103 bytes} [Thu Aug 18 16:57:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:57:39.558190 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783a0f55-fe4f-4fbd-b2ca-0ce03fe010a7 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:39.601535 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:57:39.655574 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783a0f55-fe4f-4fbd-b2ca-0ce03fe010a7 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:39.688367 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40760c2-5164-4c2e-924e-2f3989860d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:39.688989 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40760c2-5164-4c2e-924e-2f3989860d91 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:39.689269 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40760c2-5164-4c2e-924e-2f3989860d91 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:39.689539 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40760c2-5164-4c2e-924e-2f3989860d91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:39.702315 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40760c2-5164-4c2e-924e-2f3989860d91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:39.703789 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40760c2-5164-4c2e-924e-2f3989860d91 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:39.718424 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-e40760c2-5164-4c2e-924e-2f3989860d91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:57:39.718424 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:57:39.718424 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:57:39.718424 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:57:39.718424 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:57:39.718424 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:57:39.718424 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:57:39.718424 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:57:39.718424 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:57:39.718424 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:57:39.718424 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:57:39.718424 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:57:39.718424 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:57:39.718424 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:57:39.718424 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:57:39.718424 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:57:39.722459 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:57:39.722459 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:57:39.722459 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:57:39.722459 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:57:39.722459 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:57:39.722459 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:57:39.722459 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:57:39.722459 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:57:39.722459 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:57:39.722459 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:57:39.722459 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:57:39.722459 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:57:39.722459 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:57:39.722459 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:57:39.722459 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:57:39.722459 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:57:39.722459 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:57:39.724447 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:57:39.724447 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:57:39.724447 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:57:39.724447 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 615/1224] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:57:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:57:39.726112 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2555a1c-c3d3-4f6d-9c8d-8de2f035282a None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:39.828374 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2555a1c-c3d3-4f6d-9c8d-8de2f035282a None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:39.862212 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce865bd-9b90-4d4b-ab37-e1d4c38e98ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:39.862816 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce865bd-9b90-4d4b-ab37-e1d4c38e98ab None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:39.863129 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce865bd-9b90-4d4b-ab37-e1d4c38e98ab None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:39.863803 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce865bd-9b90-4d4b-ab37-e1d4c38e98ab None admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:39.864446 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce865bd-9b90-4d4b-ab37-e1d4c38e98ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:57:39.864747 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce865bd-9b90-4d4b-ab37-e1d4c38e98ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:39.866125 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce865bd-9b90-4d4b-ab37-e1d4c38e98ab None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:39.877624 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 610/1225] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:57:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:57:39.883709 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40760c2-5164-4c2e-924e-2f3989860d91 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:39.975095 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:57:39.981502 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40760c2-5164-4c2e-924e-2f3989860d91 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:40.014185 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f7e3a0-ab2b-4381-bc0b-23ee681678e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:40.014802 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f7e3a0-ab2b-4381-bc0b-23ee681678e2 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:40.015115 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f7e3a0-ab2b-4381-bc0b-23ee681678e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:40.015388 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f7e3a0-ab2b-4381-bc0b-23ee681678e2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:40.022163 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f7e3a0-ab2b-4381-bc0b-23ee681678e2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:40.023586 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f7e3a0-ab2b-4381-bc0b-23ee681678e2 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:40.035812 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-35f7e3a0-ab2b-4381-bc0b-23ee681678e2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 18 16:57:40.035812 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:57:40.035812 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:57:40.035812 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:57:40.035812 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:57:40.035812 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:57:40.035812 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:57:40.035812 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:57:40.035812 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:57:40.035812 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:57:40.035812 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:57:40.035812 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:57:40.035812 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 16:57:40.035812 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 16:57:40.035812 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 16:57:40.035812 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 16:57:40.038315 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:57:40.038315 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:57:40.038315 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 16:57:40.038315 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:57:40.038315 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 16:57:40.038315 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:57:40.038315 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:57:40.038315 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:57:40.038315 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:57:40.038315 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:57:40.038315 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:57:40.038315 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:57:40.038315 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:57:40.038315 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:57:40.038315 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:57:40.038315 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:57:40.038315 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:57:40.038315 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:57:40.039320 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:57:40.039320 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:57:40.039320 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 16:57:40.039320 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 16:57:40.039320 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 16:57:40.039320 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 16:57:40.039320 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 16:57:40.039320 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 16:57:40.039320 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 18 16:57:40.039320 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:57:40.039320 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 616/1226] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:57:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:57:40.042890 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce865bd-9b90-4d4b-ab37-e1d4c38e98ab None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:40.089396 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:57:40.142613 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce865bd-9b90-4d4b-ab37-e1d4c38e98ab None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:40.177089 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32ef727-500e-48b6-b783-2b91d0487bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:40.177686 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ef727-500e-48b6-b783-2b91d0487bdf None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:40.177962 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ef727-500e-48b6-b783-2b91d0487bdf None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:40.178232 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ef727-500e-48b6-b783-2b91d0487bdf None admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:40.178972 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32ef727-500e-48b6-b783-2b91d0487bdf None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:57:40.179274 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32ef727-500e-48b6-b783-2b91d0487bdf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:40.180759 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32ef727-500e-48b6-b783-2b91d0487bdf None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:40.197267 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 611/1227] 10.222.0.82 () {56 vars in 1143 bytes} [Thu Aug 18 16:57:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:57:40.203527 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f7e3a0-ab2b-4381-bc0b-23ee681678e2 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:40.300837 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f7e3a0-ab2b-4381-bc0b-23ee681678e2 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:40.335762 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a27e7c4-2fe6-41ef-bfb4-90cada79543b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:40.336382 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a27e7c4-2fe6-41ef-bfb4-90cada79543b None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:40.336656 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a27e7c4-2fe6-41ef-bfb4-90cada79543b None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:40.336929 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a27e7c4-2fe6-41ef-bfb4-90cada79543b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:40.347017 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a27e7c4-2fe6-41ef-bfb4-90cada79543b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:40.348562 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a27e7c4-2fe6-41ef-bfb4-90cada79543b None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:40.361978 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-6a27e7c4-2fe6-41ef-bfb4-90cada79543b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:57:40.361978 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:57:40.361978 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:57:40.361978 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:57:40.361978 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:57:40.361978 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:57:40.361978 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:57:40.361978 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:57:40.361978 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:57:40.361978 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:57:40.361978 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:57:40.361978 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:57:40.361978 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:57:40.361978 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:57:40.361978 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:57:40.361978 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:57:40.369249 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:57:40.369249 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:57:40.369249 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:57:40.369249 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:57:40.369249 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:57:40.369249 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:57:40.369249 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:57:40.369249 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:57:40.369249 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:57:40.369249 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:57:40.369249 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:57:40.369249 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:57:40.369249 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:57:40.369249 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:57:40.369249 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:57:40.369249 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:57:40.369249 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:57:40.370373 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:57:40.370373 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:57:40.370373 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:57:40.370373 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 617/1228] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:57:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:57:40.370668 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32ef727-500e-48b6-b783-2b91d0487bdf None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:40.467792 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32ef727-500e-48b6-b783-2b91d0487bdf None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:40.470041 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:57:40.501067 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da6ce9b-2df4-4192-826a-577fdf0900d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:40.501669 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da6ce9b-2df4-4192-826a-577fdf0900d6 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:40.501947 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da6ce9b-2df4-4192-826a-577fdf0900d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:40.502216 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da6ce9b-2df4-4192-826a-577fdf0900d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:40.502842 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da6ce9b-2df4-4192-826a-577fdf0900d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:57:40.503248 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da6ce9b-2df4-4192-826a-577fdf0900d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:40.505368 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da6ce9b-2df4-4192-826a-577fdf0900d6 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:40.516373 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 612/1229] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:57:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:57:40.522305 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a27e7c4-2fe6-41ef-bfb4-90cada79543b None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:40.531718 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:57:40.621055 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a27e7c4-2fe6-41ef-bfb4-90cada79543b None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:40.653663 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7ec253-31bb-462b-ba54-c5211ae19f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:40.654280 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7ec253-31bb-462b-ba54-c5211ae19f78 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:40.654553 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7ec253-31bb-462b-ba54-c5211ae19f78 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:40.654819 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7ec253-31bb-462b-ba54-c5211ae19f78 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:40.661519 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7ec253-31bb-462b-ba54-c5211ae19f78 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:40.662960 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7ec253-31bb-462b-ba54-c5211ae19f78 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:40.673632 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-ed7ec253-31bb-462b-ba54-c5211ae19f78 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:57:40.673632 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:57:40.673632 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:57:40.673632 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:57:40.673632 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:57:40.673632 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:57:40.673632 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:57:40.673632 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:57:40.673632 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:57:40.673632 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:57:40.673632 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:57:40.673632 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:57:40.673632 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 16:57:40.673632 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 16:57:40.673632 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 16:57:40.674642 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 16:57:40.674642 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:57:40.674642 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:57:40.674642 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:57:40.674642 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:57:40.674642 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:57:40.674642 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:57:40.674642 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:57:40.674642 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:57:40.674642 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:57:40.674642 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:57:40.674642 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:57:40.674642 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:57:40.674642 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:57:40.674642 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:57:40.674642 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:57:40.674642 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:57:40.676424 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 16:57:40.676424 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 16:57:40.676424 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 16:57:40.676424 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 16:57:40.676424 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 16:57:40.676424 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 16:57:40.676424 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:57:40.676424 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:57:40.676424 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 618/1230] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 16:57:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:57:40.681382 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da6ce9b-2df4-4192-826a-577fdf0900d6 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:40.780299 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da6ce9b-2df4-4192-826a-577fdf0900d6 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:40.814579 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664e895e-d663-4253-8aee-38feb70c0cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:40.815270 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664e895e-d663-4253-8aee-38feb70c0cf8 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:40.815555 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664e895e-d663-4253-8aee-38feb70c0cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:40.815827 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664e895e-d663-4253-8aee-38feb70c0cf8 None admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:40.817152 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664e895e-d663-4253-8aee-38feb70c0cf8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:57:40.817458 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664e895e-d663-4253-8aee-38feb70c0cf8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:40.818913 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664e895e-d663-4253-8aee-38feb70c0cf8 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:40.833871 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 613/1231] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 16:57:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:57:40.839980 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7ec253-31bb-462b-ba54-c5211ae19f78 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:40.903911 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:57:40.937217 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7ec253-31bb-462b-ba54-c5211ae19f78 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:40.968646 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82845f46-9861-46cd-bfde-248da700c6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:40.969379 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82845f46-9861-46cd-bfde-248da700c6c0 None admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:40.969657 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82845f46-9861-46cd-bfde-248da700c6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:40.969926 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82845f46-9861-46cd-bfde-248da700c6c0 None admin] PATH_INFO: `/v3/projects/2293c0aab61f44849d0d48b214cde7ea/users/29653ba1a2a74232972ef7a9000518b0/roles/5e0f402e1af84b44a723ee0730315d74` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:40.984103 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82845f46-9861-46cd-bfde-248da700c6c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=2293c0aab61f44849d0d48b214cde7ea, user_id=29653ba1a2a74232972ef7a9000518b0, role_id=5e0f402e1af84b44a723ee0730315d74)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:40.985520 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82845f46-9861-46cd-bfde-248da700c6c0 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:41.049915 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 619/1232] 10.222.0.82 () {58 vars in 1439 bytes} [Thu Aug 18 16:57:40 2022] PUT /identity/v3/projects/2293c0aab61f44849d0d48b214cde7ea/users/29653ba1a2a74232972ef7a9000518b0/roles/5e0f402e1af84b44a723ee0730315d74 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 16:57:42.942806 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceaed5e-1583-4e58-ad7d-6e5aed9f61d3 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:42.943408 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceaed5e-1583-4e58-ad7d-6e5aed9f61d3 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:42.943408 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceaed5e-1583-4e58-ad7d-6e5aed9f61d3 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:42.944783 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 614/1233] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:57:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:57:42.950023 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274327ce-b327-4ff2-af7c-19e7f3415979 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:42.955266 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274327ce-b327-4ff2-af7c-19e7f3415979 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:42.955266 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274327ce-b327-4ff2-af7c-19e7f3415979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:42.969216 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-274327ce-b327-4ff2-af7c-19e7f3415979 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:57:42.972525 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-274327ce-b327-4ff2-af7c-19e7f3415979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:57:43.036810 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 620/1234] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:57:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 16:57:43.311231 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceaed5e-1583-4e58-ad7d-6e5aed9f61d3 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:43.337660 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:57:43.411179 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceaed5e-1583-4e58-ad7d-6e5aed9f61d3 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:43.444301 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6750542-1031-4403-bdeb-c45c37084315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:43.444925 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6750542-1031-4403-bdeb-c45c37084315 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:43.445202 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6750542-1031-4403-bdeb-c45c37084315 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:43.445470 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6750542-1031-4403-bdeb-c45c37084315 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:43.455679 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6750542-1031-4403-bdeb-c45c37084315 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:57:43.461033 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-f6750542-1031-4403-bdeb-c45c37084315 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 16:57:43.461033 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:57:43.461033 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:57:43.461033 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:57:43.461033 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:57:43.461033 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:57:43.461033 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:57:43.461033 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:57:43.461033 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:57:43.461033 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:57:43.461033 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:57:43.461033 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:57:43.461033 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:57:43.461033 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:57:43.461033 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:57:43.461033 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:57:43.463107 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:57:43.463107 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:57:43.463107 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:57:43.463107 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:57:43.463107 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:57:43.463107 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:57:43.463107 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:57:43.463107 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:57:43.463107 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:57:43.463107 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:57:43.463107 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:57:43.463107 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:57:43.463107 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:57:43.463107 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:57:43.463107 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:57:43.463107 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:57:43.464206 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:57:43.464206 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:57:43.464206 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:57:43.464206 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:57:43.464206 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:57:43.464206 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:57:43.464206 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:57:43.464206 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:57:43.464206 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:57:43.464206 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:57:43.464206 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 16:57:43.464206 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:57:43.464206 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 615/1235] 10.222.0.82 () {56 vars in 1102 bytes} [Thu Aug 18 16:57:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:57:43.468517 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274327ce-b327-4ff2-af7c-19e7f3415979 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:43.514905 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:57:43.566084 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274327ce-b327-4ff2-af7c-19e7f3415979 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:43.599283 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2c410b-4916-4063-a677-8cbc31199fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:43.599881 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2c410b-4916-4063-a677-8cbc31199fec None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:43.600157 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2c410b-4916-4063-a677-8cbc31199fec None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:43.600424 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2c410b-4916-4063-a677-8cbc31199fec None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:43.601069 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2c410b-4916-4063-a677-8cbc31199fec None admin] RBAC: Adding query filter params (name=service) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:57:43.601367 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2c410b-4916-4063-a677-8cbc31199fec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:43.602729 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2c410b-4916-4063-a677-8cbc31199fec None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:43.610174 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 621/1236] 10.222.0.82 () {56 vars in 1103 bytes} [Thu Aug 18 16:57:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:57:43.616214 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6750542-1031-4403-bdeb-c45c37084315 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:43.701937 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:57:43.718233 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6750542-1031-4403-bdeb-c45c37084315 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:43.750694 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036ea2e4-6dfd-4277-a242-368580df9372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:43.751342 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ea2e4-6dfd-4277-a242-368580df9372 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:43.751622 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ea2e4-6dfd-4277-a242-368580df9372 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:43.751888 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ea2e4-6dfd-4277-a242-368580df9372 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:43.762034 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036ea2e4-6dfd-4277-a242-368580df9372 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:43.763510 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036ea2e4-6dfd-4277-a242-368580df9372 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:43.777662 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-036ea2e4-6dfd-4277-a242-368580df9372 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:57:43.777662 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:57:43.777662 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:57:43.777662 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:57:43.777662 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:57:43.777662 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:57:43.777662 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:57:43.777662 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:57:43.777662 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:57:43.777662 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:57:43.777662 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:57:43.777662 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:57:43.777662 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:57:43.777662 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:57:43.777662 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:57:43.777662 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:57:43.778857 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:57:43.778857 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:57:43.778857 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:57:43.778857 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:57:43.778857 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:57:43.778857 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:57:43.778857 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:57:43.778857 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:57:43.778857 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:57:43.778857 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:57:43.778857 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:57:43.778857 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:57:43.778857 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:57:43.778857 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:57:43.778857 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:57:43.778857 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:57:43.778857 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:57:43.779797 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:57:43.779797 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:57:43.779797 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:57:43.779797 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 616/1237] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:57:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:57:43.784695 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2c410b-4916-4063-a677-8cbc31199fec None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:43.882361 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2c410b-4916-4063-a677-8cbc31199fec None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:43.890805 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:57:43.914918 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bec69d-551e-43bd-8a6d-0373ae30cb45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:43.915519 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bec69d-551e-43bd-8a6d-0373ae30cb45 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:43.915799 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bec69d-551e-43bd-8a6d-0373ae30cb45 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:43.916070 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bec69d-551e-43bd-8a6d-0373ae30cb45 None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:43.916698 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bec69d-551e-43bd-8a6d-0373ae30cb45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:57:43.916997 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bec69d-551e-43bd-8a6d-0373ae30cb45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:43.918354 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bec69d-551e-43bd-8a6d-0373ae30cb45 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:43.929649 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 622/1238] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:57:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:57:43.935390 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036ea2e4-6dfd-4277-a242-368580df9372 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:44.032360 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036ea2e4-6dfd-4277-a242-368580df9372 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:44.064038 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9a84f9-44cb-4c3c-b99f-a2755d3b0b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:44.064655 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9a84f9-44cb-4c3c-b99f-a2755d3b0b21 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:44.064933 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9a84f9-44cb-4c3c-b99f-a2755d3b0b21 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:44.065204 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9a84f9-44cb-4c3c-b99f-a2755d3b0b21 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:44.068406 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:57:44.073447 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9a84f9-44cb-4c3c-b99f-a2755d3b0b21 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:44.074891 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9a84f9-44cb-4c3c-b99f-a2755d3b0b21 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:44.086429 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-4e9a84f9-44cb-4c3c-b99f-a2755d3b0b21 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 18 16:57:44.086429 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:57:44.086429 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:57:44.086429 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:57:44.086429 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:57:44.086429 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:57:44.086429 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:57:44.086429 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:57:44.086429 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:57:44.086429 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:57:44.086429 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:57:44.086429 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:57:44.086429 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 16:57:44.086429 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 16:57:44.086429 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 16:57:44.086429 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 16:57:44.087579 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:57:44.087579 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:57:44.087579 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 16:57:44.087579 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:57:44.087579 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 16:57:44.087579 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:57:44.087579 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:57:44.087579 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:57:44.087579 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:57:44.087579 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:57:44.087579 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:57:44.087579 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:57:44.087579 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:57:44.087579 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:57:44.087579 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:57:44.087579 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:57:44.087579 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:57:44.087579 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:57:44.088809 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:57:44.088809 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:57:44.088809 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 16:57:44.088809 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 16:57:44.088809 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 16:57:44.088809 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 16:57:44.088809 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 16:57:44.088809 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 16:57:44.088809 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 18 16:57:44.088809 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:57:44.091073 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 617/1239] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:57:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:57:44.093302 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bec69d-551e-43bd-8a6d-0373ae30cb45 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:44.192450 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bec69d-551e-43bd-8a6d-0373ae30cb45 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:44.225480 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87327af7-f630-4b4c-8e2a-e2527a3971c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:44.226122 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87327af7-f630-4b4c-8e2a-e2527a3971c1 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:44.226397 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87327af7-f630-4b4c-8e2a-e2527a3971c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:44.226668 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87327af7-f630-4b4c-8e2a-e2527a3971c1 None admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:44.227408 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87327af7-f630-4b4c-8e2a-e2527a3971c1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:57:44.227710 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87327af7-f630-4b4c-8e2a-e2527a3971c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:44.229148 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87327af7-f630-4b4c-8e2a-e2527a3971c1 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:44.246179 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 623/1240] 10.222.0.82 () {56 vars in 1143 bytes} [Thu Aug 18 16:57:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:57:44.252072 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9a84f9-44cb-4c3c-b99f-a2755d3b0b21 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:44.355282 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9a84f9-44cb-4c3c-b99f-a2755d3b0b21 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:44.387294 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed875e00-ce16-4dc9-9ebb-0a9c21a5e29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:44.387899 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed875e00-ce16-4dc9-9ebb-0a9c21a5e29f None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:44.388180 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed875e00-ce16-4dc9-9ebb-0a9c21a5e29f None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:44.388454 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed875e00-ce16-4dc9-9ebb-0a9c21a5e29f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:44.448520 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:57:44.455062 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed875e00-ce16-4dc9-9ebb-0a9c21a5e29f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:44.456614 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed875e00-ce16-4dc9-9ebb-0a9c21a5e29f None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:44.552638 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 618/1241] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:57:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 16:57:44.558483 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87327af7-f630-4b4c-8e2a-e2527a3971c1 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:44.586415 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:57:44.655698 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87327af7-f630-4b4c-8e2a-e2527a3971c1 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:44.688232 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b83dbb-ea16-4f34-9494-3396d42a6ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:44.688856 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b83dbb-ea16-4f34-9494-3396d42a6ea5 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:44.689136 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b83dbb-ea16-4f34-9494-3396d42a6ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:44.689408 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b83dbb-ea16-4f34-9494-3396d42a6ea5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:44.699842 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b83dbb-ea16-4f34-9494-3396d42a6ea5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:44.701485 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b83dbb-ea16-4f34-9494-3396d42a6ea5 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:44.716205 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-96b83dbb-ea16-4f34-9494-3396d42a6ea5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:57:44.716205 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:57:44.716205 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:57:44.716205 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:57:44.716205 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:57:44.716205 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:57:44.716205 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:57:44.716205 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:57:44.716205 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:57:44.716205 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:57:44.716205 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:57:44.716205 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:57:44.716205 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:57:44.716205 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:57:44.716205 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:57:44.716205 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:57:44.717138 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:57:44.717138 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:57:44.717138 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:57:44.717138 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:57:44.717138 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:57:44.717138 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:57:44.717138 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:57:44.717138 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:57:44.717138 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:57:44.717138 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:57:44.717138 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:57:44.717138 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:57:44.717138 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:57:44.717138 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:57:44.717138 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:57:44.717138 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:57:44.717138 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:57:44.718258 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:57:44.718258 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:57:44.718258 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:57:44.718258 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 624/1242] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:57:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:57:44.723535 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed875e00-ce16-4dc9-9ebb-0a9c21a5e29f None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:44.815254 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:57:44.821793 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed875e00-ce16-4dc9-9ebb-0a9c21a5e29f None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:44.856832 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a427c0-65fd-4924-b903-5947c3777519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:44.857434 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a427c0-65fd-4924-b903-5947c3777519 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:44.857714 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a427c0-65fd-4924-b903-5947c3777519 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:44.857985 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a427c0-65fd-4924-b903-5947c3777519 None admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:44.858615 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a427c0-65fd-4924-b903-5947c3777519 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:57:44.858947 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a427c0-65fd-4924-b903-5947c3777519 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:44.860326 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a427c0-65fd-4924-b903-5947c3777519 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:44.871524 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 619/1243] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:57:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:57:44.877216 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b83dbb-ea16-4f34-9494-3396d42a6ea5 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:44.955441 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:57:44.977588 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b83dbb-ea16-4f34-9494-3396d42a6ea5 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:45.013928 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d03917-2462-40c7-9cea-4b045907692f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:45.014548 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d03917-2462-40c7-9cea-4b045907692f None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:45.014827 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d03917-2462-40c7-9cea-4b045907692f None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:45.015331 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d03917-2462-40c7-9cea-4b045907692f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:45.026375 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d03917-2462-40c7-9cea-4b045907692f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:45.028324 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d03917-2462-40c7-9cea-4b045907692f None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:45.040201 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-09d03917-2462-40c7-9cea-4b045907692f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:57:45.040201 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:57:45.040201 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:57:45.040201 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:57:45.040201 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:57:45.040201 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:57:45.040201 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:57:45.040201 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:57:45.040201 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:57:45.040201 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:57:45.040201 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:57:45.040201 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:57:45.040201 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 16:57:45.040201 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 16:57:45.040201 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 16:57:45.041259 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 16:57:45.041259 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:57:45.041259 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:57:45.041259 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:57:45.041259 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:57:45.041259 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:57:45.041259 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:57:45.041259 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:57:45.041259 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:57:45.041259 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:57:45.041259 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:57:45.041259 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:57:45.041259 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:57:45.041259 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:57:45.041259 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:57:45.041259 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:57:45.041259 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:57:45.042169 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 16:57:45.042169 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 16:57:45.042169 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 16:57:45.042169 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 16:57:45.042169 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 16:57:45.042169 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 16:57:45.042169 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:57:45.042169 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:57:45.042169 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 625/1244] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 16:57:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:57:45.047256 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a427c0-65fd-4924-b903-5947c3777519 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:45.145994 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a427c0-65fd-4924-b903-5947c3777519 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:45.179172 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5df949-3bf6-459f-8fa5-7cf61e771337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:45.179762 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5df949-3bf6-459f-8fa5-7cf61e771337 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:45.180039 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5df949-3bf6-459f-8fa5-7cf61e771337 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:45.180305 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5df949-3bf6-459f-8fa5-7cf61e771337 None admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:45.180982 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5df949-3bf6-459f-8fa5-7cf61e771337 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:57:45.181284 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5df949-3bf6-459f-8fa5-7cf61e771337 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:45.182713 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5df949-3bf6-459f-8fa5-7cf61e771337 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:45.194621 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 620/1245] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 16:57:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:57:45.201101 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d03917-2462-40c7-9cea-4b045907692f None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:45.300178 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d03917-2462-40c7-9cea-4b045907692f None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:45.334102 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d28cd70-d606-413f-93b8-17874984a219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:45.334761 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d28cd70-d606-413f-93b8-17874984a219 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:45.335075 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d28cd70-d606-413f-93b8-17874984a219 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:45.335352 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d28cd70-d606-413f-93b8-17874984a219 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:45.336182 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d28cd70-d606-413f-93b8-17874984a219 None admin] RBAC: Adding query filter params (role.id=5e0f402e1af84b44a723ee0730315d74, scope.project.id=2293c0aab61f44849d0d48b214cde7ea, user.id=29653ba1a2a74232972ef7a9000518b0) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:57:45.336491 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d28cd70-d606-413f-93b8-17874984a219 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:45.337936 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d28cd70-d606-413f-93b8-17874984a219 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:45.343856 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 626/1246] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 16:57:45 2022] GET /identity/v3/role_assignments?user.id=29653ba1a2a74232972ef7a9000518b0&scope.project.id=2293c0aab61f44849d0d48b214cde7ea&role.id=5e0f402e1af84b44a723ee0730315d74 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:57:47.332260 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9867c-8046-4d73-ba48-3682a8c6d0c3 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:47.332816 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9867c-8046-4d73-ba48-3682a8c6d0c3 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:47.332816 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9867c-8046-4d73-ba48-3682a8c6d0c3 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:47.334212 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 621/1247] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:57:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:57:47.339554 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d4111-500d-4e1a-9f24-353b9ba679b9 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:47.340046 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d4111-500d-4e1a-9f24-353b9ba679b9 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:47.340153 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d4111-500d-4e1a-9f24-353b9ba679b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:47.346442 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:57:47.358879 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-a61d4111-500d-4e1a-9f24-353b9ba679b9 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:57:47.360200 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-a61d4111-500d-4e1a-9f24-353b9ba679b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:57:47.428090 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 627/1248] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:57:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 16:57:47.693693 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c9867c-8046-4d73-ba48-3682a8c6d0c3 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:47.712300 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:57:47.795793 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c9867c-8046-4d73-ba48-3682a8c6d0c3 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:47.828867 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36fa6c0-e673-4ce8-8ed3-eb1a20612110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:47.829613 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36fa6c0-e673-4ce8-8ed3-eb1a20612110 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:47.829900 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36fa6c0-e673-4ce8-8ed3-eb1a20612110 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:47.830172 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36fa6c0-e673-4ce8-8ed3-eb1a20612110 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:47.840995 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36fa6c0-e673-4ce8-8ed3-eb1a20612110 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:57:47.845630 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-e36fa6c0-e673-4ce8-8ed3-eb1a20612110 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 16:57:47.845630 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:57:47.845630 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:57:47.845630 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:57:47.845630 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:57:47.845630 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:57:47.845630 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:57:47.845630 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:57:47.845630 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:57:47.845630 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:57:47.845630 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:57:47.845630 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:57:47.845630 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:57:47.845630 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:57:47.845630 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:57:47.845630 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:57:47.846703 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:57:47.846703 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:57:47.846703 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:57:47.846703 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:57:47.846703 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:57:47.846703 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:57:47.846703 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:57:47.846703 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:57:47.846703 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:57:47.846703 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:57:47.846703 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:57:47.846703 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:57:47.846703 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:57:47.846703 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:57:47.846703 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:57:47.846703 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:57:47.848064 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:57:47.848064 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:57:47.848064 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:57:47.848064 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:57:47.848064 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:57:47.848064 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:57:47.848064 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:57:47.848064 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:57:47.848064 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:57:47.848064 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:57:47.848064 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 16:57:47.848064 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:57:47.848064 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 622/1249] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 16:57:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 16:57:47.852916 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61d4111-500d-4e1a-9f24-353b9ba679b9 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:47.948627 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61d4111-500d-4e1a-9f24-353b9ba679b9 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:47.977164 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:57:47.981845 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88a99b8-76a1-4e24-9129-0f318019287e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:47.982444 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88a99b8-76a1-4e24-9129-0f318019287e None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:47.982718 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88a99b8-76a1-4e24-9129-0f318019287e None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:47.983017 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88a99b8-76a1-4e24-9129-0f318019287e None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:47.983672 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88a99b8-76a1-4e24-9129-0f318019287e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:57:47.983968 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88a99b8-76a1-4e24-9129-0f318019287e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:47.985310 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88a99b8-76a1-4e24-9129-0f318019287e None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:47.993550 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 628/1250] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:57:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:57:47.999427 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36fa6c0-e673-4ce8-8ed3-eb1a20612110 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:48.078314 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:57:48.101179 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36fa6c0-e673-4ce8-8ed3-eb1a20612110 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:48.135042 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4ebfeb-046f-4831-b171-7af64137df01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:48.135663 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4ebfeb-046f-4831-b171-7af64137df01 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:48.135941 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4ebfeb-046f-4831-b171-7af64137df01 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:48.136210 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4ebfeb-046f-4831-b171-7af64137df01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:48.146329 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4ebfeb-046f-4831-b171-7af64137df01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:48.148032 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4ebfeb-046f-4831-b171-7af64137df01 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:48.162126 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-bd4ebfeb-046f-4831-b171-7af64137df01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:57:48.162126 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:57:48.162126 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:57:48.162126 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:57:48.162126 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:57:48.162126 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:57:48.162126 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:57:48.162126 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:57:48.162126 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:57:48.162126 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:57:48.162126 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:57:48.162126 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:57:48.162126 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:57:48.162126 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:57:48.162126 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:57:48.162126 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:57:48.163224 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:57:48.163224 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:57:48.163224 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:57:48.163224 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:57:48.163224 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:57:48.163224 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:57:48.163224 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:57:48.163224 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:57:48.163224 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:57:48.163224 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:57:48.163224 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:57:48.163224 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:57:48.163224 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:57:48.163224 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:57:48.163224 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:57:48.163224 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:57:48.163224 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:57:48.164246 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:57:48.164246 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:57:48.164246 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:57:48.164246 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 623/1251] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:57:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:57:48.169424 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88a99b8-76a1-4e24-9129-0f318019287e None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:48.267318 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88a99b8-76a1-4e24-9129-0f318019287e None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:48.299487 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d810f75b-2bd7-472c-9f26-5a02ad585e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:48.300583 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d810f75b-2bd7-472c-9f26-5a02ad585e58 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:48.300863 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d810f75b-2bd7-472c-9f26-5a02ad585e58 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:48.301165 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d810f75b-2bd7-472c-9f26-5a02ad585e58 None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:48.301824 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d810f75b-2bd7-472c-9f26-5a02ad585e58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:57:48.302121 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d810f75b-2bd7-472c-9f26-5a02ad585e58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:48.303601 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d810f75b-2bd7-472c-9f26-5a02ad585e58 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:48.314827 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 629/1252] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:57:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:57:48.321568 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4ebfeb-046f-4831-b171-7af64137df01 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:48.421683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4ebfeb-046f-4831-b171-7af64137df01 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:48.447622 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:57:48.455449 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60526042-82fa-4156-8dab-2d7a1a064459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:48.456077 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60526042-82fa-4156-8dab-2d7a1a064459 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:48.456482 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60526042-82fa-4156-8dab-2d7a1a064459 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:48.456763 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60526042-82fa-4156-8dab-2d7a1a064459 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:48.465024 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60526042-82fa-4156-8dab-2d7a1a064459 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:48.470309 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60526042-82fa-4156-8dab-2d7a1a064459 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:48.477881 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-60526042-82fa-4156-8dab-2d7a1a064459 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 18 16:57:48.477881 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:57:48.477881 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:57:48.477881 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:57:48.477881 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:57:48.477881 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:57:48.477881 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:57:48.477881 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:57:48.477881 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:57:48.477881 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:57:48.477881 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:57:48.477881 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:57:48.477881 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 16:57:48.477881 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 16:57:48.477881 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 16:57:48.477881 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 16:57:48.479085 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:57:48.479085 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:57:48.479085 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 16:57:48.479085 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:57:48.479085 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 16:57:48.479085 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:57:48.479085 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:57:48.479085 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:57:48.479085 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:57:48.479085 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:57:48.479085 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:57:48.479085 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:57:48.479085 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:57:48.479085 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:57:48.479085 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:57:48.479085 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:57:48.479085 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:57:48.479085 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:57:48.480445 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:57:48.480445 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:57:48.480445 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 16:57:48.480445 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 16:57:48.480445 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 16:57:48.480445 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 16:57:48.480445 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 16:57:48.480445 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 16:57:48.480445 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 18 16:57:48.480445 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:57:48.480445 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 624/1253] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:57:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:57:48.484855 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d810f75b-2bd7-472c-9f26-5a02ad585e58 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:48.540753 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:57:48.584130 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d810f75b-2bd7-472c-9f26-5a02ad585e58 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:48.618094 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefe852b-7098-4645-b521-53289fa3ab75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:48.618692 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe852b-7098-4645-b521-53289fa3ab75 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:48.619002 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe852b-7098-4645-b521-53289fa3ab75 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:48.619278 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe852b-7098-4645-b521-53289fa3ab75 None admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:48.620015 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefe852b-7098-4645-b521-53289fa3ab75 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:57:48.620315 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefe852b-7098-4645-b521-53289fa3ab75 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:48.621727 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefe852b-7098-4645-b521-53289fa3ab75 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:48.638289 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 630/1254] 10.222.0.82 () {56 vars in 1143 bytes} [Thu Aug 18 16:57:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:57:48.644309 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60526042-82fa-4156-8dab-2d7a1a064459 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:48.744003 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60526042-82fa-4156-8dab-2d7a1a064459 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:48.777468 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e303d47-e099-44ec-940d-9d8658f35579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:48.778079 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e303d47-e099-44ec-940d-9d8658f35579 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:48.778362 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e303d47-e099-44ec-940d-9d8658f35579 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:48.778636 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e303d47-e099-44ec-940d-9d8658f35579 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:48.833970 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:57:48.845638 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e303d47-e099-44ec-940d-9d8658f35579 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:48.848005 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e303d47-e099-44ec-940d-9d8658f35579 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:48.944639 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 625/1255] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:57:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 16:57:48.950867 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefe852b-7098-4645-b521-53289fa3ab75 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:49.047809 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefe852b-7098-4645-b521-53289fa3ab75 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:49.061143 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:57:49.083483 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d83223-1984-4e1c-bdce-56e3d6c5463c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:49.084225 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d83223-1984-4e1c-bdce-56e3d6c5463c None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:49.084455 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d83223-1984-4e1c-bdce-56e3d6c5463c None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:49.084721 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d83223-1984-4e1c-bdce-56e3d6c5463c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:49.098813 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d83223-1984-4e1c-bdce-56e3d6c5463c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:49.100312 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d83223-1984-4e1c-bdce-56e3d6c5463c None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:49.115031 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-90d83223-1984-4e1c-bdce-56e3d6c5463c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:57:49.115031 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:57:49.115031 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:57:49.115031 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:57:49.115031 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:57:49.115031 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:57:49.115031 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:57:49.115031 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:57:49.115031 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:57:49.115031 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:57:49.115031 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:57:49.115031 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:57:49.115031 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:57:49.115031 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:57:49.115031 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:57:49.115031 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:57:49.116162 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:57:49.116162 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:57:49.116162 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:57:49.116162 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:57:49.116162 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:57:49.116162 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:57:49.116162 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:57:49.116162 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:57:49.116162 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:57:49.116162 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:57:49.116162 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:57:49.116162 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:57:49.116162 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:57:49.116162 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:57:49.116162 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:57:49.116162 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:57:49.116162 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:57:49.117206 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:57:49.117206 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:57:49.117206 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:57:49.117206 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 631/1256] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:57:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:57:49.121981 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e303d47-e099-44ec-940d-9d8658f35579 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:49.208003 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:57:49.219433 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e303d47-e099-44ec-940d-9d8658f35579 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:49.252543 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c297e2a9-6f55-471d-842e-6639f2f52052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:49.253174 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c297e2a9-6f55-471d-842e-6639f2f52052 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:49.253498 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c297e2a9-6f55-471d-842e-6639f2f52052 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:49.253763 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c297e2a9-6f55-471d-842e-6639f2f52052 None admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:49.254405 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c297e2a9-6f55-471d-842e-6639f2f52052 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:57:49.254687 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c297e2a9-6f55-471d-842e-6639f2f52052 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:49.256083 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c297e2a9-6f55-471d-842e-6639f2f52052 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:49.267119 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 626/1257] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:57:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:57:49.272884 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d83223-1984-4e1c-bdce-56e3d6c5463c None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:49.373621 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d83223-1984-4e1c-bdce-56e3d6c5463c None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:49.408063 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc90fc1-e80a-4ade-bfef-d1020efc4cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:49.408687 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc90fc1-e80a-4ade-bfef-d1020efc4cff None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:49.408965 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc90fc1-e80a-4ade-bfef-d1020efc4cff None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:49.409235 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc90fc1-e80a-4ade-bfef-d1020efc4cff None admin] PATH_INFO: `/v3/projects/service` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:49.415704 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc90fc1-e80a-4ade-bfef-d1020efc4cff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:49.417139 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc90fc1-e80a-4ade-bfef-d1020efc4cff None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:49.428696 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-9cc90fc1-e80a-4ade-bfef-d1020efc4cff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:57:49.428696 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:57:49.428696 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:57:49.428696 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:57:49.428696 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:57:49.428696 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:57:49.428696 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:57:49.428696 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:57:49.428696 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:57:49.428696 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:57:49.428696 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:57:49.428696 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:57:49.428696 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 16:57:49.428696 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 16:57:49.428696 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 16:57:49.429742 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 16:57:49.429742 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:57:49.429742 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:57:49.429742 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:57:49.429742 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:57:49.429742 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:57:49.429742 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:57:49.429742 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:57:49.429742 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:57:49.429742 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:57:49.429742 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:57:49.429742 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:57:49.429742 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:57:49.429742 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:57:49.429742 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:57:49.429742 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:57:49.429742 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:57:49.430781 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 16:57:49.430781 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 16:57:49.430781 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 16:57:49.430781 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 16:57:49.430781 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 16:57:49.430781 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 16:57:49.430781 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:57:49.430781 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:57:49.430781 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 632/1258] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 16:57:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:57:49.435711 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c297e2a9-6f55-471d-842e-6639f2f52052 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:49.537414 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c297e2a9-6f55-471d-842e-6639f2f52052 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:49.570170 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34eb05b-1c60-4593-b881-42073bf8facc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:49.570776 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34eb05b-1c60-4593-b881-42073bf8facc None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:49.571091 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34eb05b-1c60-4593-b881-42073bf8facc None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:49.571375 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34eb05b-1c60-4593-b881-42073bf8facc None admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:49.572088 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34eb05b-1c60-4593-b881-42073bf8facc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:57:49.572367 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34eb05b-1c60-4593-b881-42073bf8facc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:49.573804 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34eb05b-1c60-4593-b881-42073bf8facc None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:49.586427 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 627/1259] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 16:57:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:57:49.592569 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc90fc1-e80a-4ade-bfef-d1020efc4cff None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:49.618297 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:57:49.690322 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc90fc1-e80a-4ade-bfef-d1020efc4cff None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:49.724012 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1501f30-ba4a-4d85-ae7f-4766d563a548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:49.724607 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1501f30-ba4a-4d85-ae7f-4766d563a548 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:49.724887 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1501f30-ba4a-4d85-ae7f-4766d563a548 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:49.725159 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1501f30-ba4a-4d85-ae7f-4766d563a548 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:49.725871 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1501f30-ba4a-4d85-ae7f-4766d563a548 None admin] RBAC: Adding query filter params (role.id=2c6966cfb10f4d72b0242e9fb5e13b17, scope.project.id=2293c0aab61f44849d0d48b214cde7ea, user.id=29653ba1a2a74232972ef7a9000518b0) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:57:49.726182 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1501f30-ba4a-4d85-ae7f-4766d563a548 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:49.727617 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1501f30-ba4a-4d85-ae7f-4766d563a548 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:49.732841 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 633/1260] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 16:57:49 2022] GET /identity/v3/role_assignments?user.id=29653ba1a2a74232972ef7a9000518b0&scope.project.id=2293c0aab61f44849d0d48b214cde7ea&role.id=2c6966cfb10f4d72b0242e9fb5e13b17 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:57:51.618509 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc8f5e-cb1b-444b-8ca8-c7bc3aa105c6 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:51.619106 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc8f5e-cb1b-444b-8ca8-c7bc3aa105c6 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:51.619106 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc8f5e-cb1b-444b-8ca8-c7bc3aa105c6 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:51.620398 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 628/1261] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:57:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:57:51.625595 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51913d49-88f4-4458-8359-70a5a4a81004 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:51.626032 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51913d49-88f4-4458-8359-70a5a4a81004 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:51.626126 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51913d49-88f4-4458-8359-70a5a4a81004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:51.645692 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-51913d49-88f4-4458-8359-70a5a4a81004 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:57:51.646924 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-51913d49-88f4-4458-8359-70a5a4a81004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:57:51.714227 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 634/1262] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:57:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 16:57:51.984353 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc8f5e-cb1b-444b-8ca8-c7bc3aa105c6 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:51.996001 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:57:52.086941 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc8f5e-cb1b-444b-8ca8-c7bc3aa105c6 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:52.122591 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ff591a-084c-4754-891d-91f7666f8b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:52.123252 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff591a-084c-4754-891d-91f7666f8b46 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:52.123536 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff591a-084c-4754-891d-91f7666f8b46 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:52.123808 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff591a-084c-4754-891d-91f7666f8b46 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:52.133839 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ff591a-084c-4754-891d-91f7666f8b46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:57:52.138560 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-e2ff591a-084c-4754-891d-91f7666f8b46 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 16:57:52.138560 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:57:52.138560 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:57:52.138560 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:57:52.138560 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:57:52.138560 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:57:52.138560 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:57:52.138560 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:57:52.138560 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:57:52.138560 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:57:52.138560 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:57:52.138560 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:57:52.138560 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:57:52.138560 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:57:52.138560 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:57:52.138560 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:57:52.139650 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:57:52.139650 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:57:52.139650 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:57:52.139650 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:57:52.139650 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:57:52.139650 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:57:52.139650 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:57:52.139650 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:57:52.139650 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:57:52.139650 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:57:52.139650 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:57:52.139650 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:57:52.139650 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:57:52.139650 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:57:52.139650 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:57:52.139650 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:57:52.140877 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:57:52.140877 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:57:52.140877 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:57:52.140877 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:57:52.140877 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:57:52.140877 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:57:52.140877 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:57:52.140877 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:57:52.140877 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:57:52.140877 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:57:52.140877 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 16:57:52.140877 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:57:52.140877 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 629/1263] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 16:57:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:57:52.145935 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51913d49-88f4-4458-8359-70a5a4a81004 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:52.164522 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:57:52.246570 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51913d49-88f4-4458-8359-70a5a4a81004 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:52.279865 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc2a318-51e9-4814-be4d-a981db4295de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:52.280490 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc2a318-51e9-4814-be4d-a981db4295de None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:52.280763 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc2a318-51e9-4814-be4d-a981db4295de None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:52.281028 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc2a318-51e9-4814-be4d-a981db4295de None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:52.281682 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc2a318-51e9-4814-be4d-a981db4295de None admin] RBAC: Adding query filter params (name=admin) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:57:52.281979 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc2a318-51e9-4814-be4d-a981db4295de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:52.283360 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc2a318-51e9-4814-be4d-a981db4295de None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:52.291868 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 635/1264] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:57:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:57:52.297661 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ff591a-084c-4754-891d-91f7666f8b46 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:52.363780 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:57:52.397028 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ff591a-084c-4754-891d-91f7666f8b46 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:52.430085 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcbad5f-4a0e-4689-9598-ad164aa9f596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:52.430715 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcbad5f-4a0e-4689-9598-ad164aa9f596 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:52.431026 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcbad5f-4a0e-4689-9598-ad164aa9f596 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:52.431302 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcbad5f-4a0e-4689-9598-ad164aa9f596 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:52.442220 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcbad5f-4a0e-4689-9598-ad164aa9f596 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:52.443993 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcbad5f-4a0e-4689-9598-ad164aa9f596 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:52.459477 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-9dcbad5f-4a0e-4689-9598-ad164aa9f596 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:57:52.459477 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:57:52.459477 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:57:52.459477 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:57:52.459477 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:57:52.459477 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:57:52.459477 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:57:52.459477 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:57:52.459477 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:57:52.459477 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:57:52.459477 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:57:52.459477 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:57:52.459477 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:57:52.459477 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:57:52.459477 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:57:52.459477 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:57:52.460549 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:57:52.460549 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:57:52.460549 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:57:52.460549 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:57:52.460549 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:57:52.460549 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:57:52.460549 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:57:52.460549 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:57:52.460549 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:57:52.460549 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:57:52.460549 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:57:52.460549 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:57:52.460549 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:57:52.460549 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:57:52.460549 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:57:52.460549 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:57:52.460549 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:57:52.461601 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:57:52.461601 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:57:52.461601 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:57:52.461601 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 630/1265] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:57:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:57:52.466349 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc2a318-51e9-4814-be4d-a981db4295de None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:52.547933 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:57:52.567031 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc2a318-51e9-4814-be4d-a981db4295de None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:52.601774 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b697ff23-f3fd-4dce-b9c1-515d54372d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:52.602387 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b697ff23-f3fd-4dce-b9c1-515d54372d4c None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:52.602666 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b697ff23-f3fd-4dce-b9c1-515d54372d4c None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:52.602965 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b697ff23-f3fd-4dce-b9c1-515d54372d4c None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:52.603667 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b697ff23-f3fd-4dce-b9c1-515d54372d4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:57:52.603971 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b697ff23-f3fd-4dce-b9c1-515d54372d4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:52.605408 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b697ff23-f3fd-4dce-b9c1-515d54372d4c None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:52.616528 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 636/1266] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:57:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:57:52.622554 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcbad5f-4a0e-4689-9598-ad164aa9f596 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:52.723839 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcbad5f-4a0e-4689-9598-ad164aa9f596 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:52.743366 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:57:52.757230 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d9f18a-1125-4c7b-b812-d17f86168a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:52.757858 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d9f18a-1125-4c7b-b812-d17f86168a37 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:52.758142 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d9f18a-1125-4c7b-b812-d17f86168a37 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:52.758416 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d9f18a-1125-4c7b-b812-d17f86168a37 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:52.765673 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d9f18a-1125-4c7b-b812-d17f86168a37 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:52.767235 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d9f18a-1125-4c7b-b812-d17f86168a37 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:52.779285 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-c0d9f18a-1125-4c7b-b812-d17f86168a37 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 18 16:57:52.779285 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:57:52.779285 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:57:52.779285 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:57:52.779285 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:57:52.779285 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:57:52.779285 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:57:52.779285 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:57:52.779285 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:57:52.779285 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:57:52.779285 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:57:52.779285 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:57:52.779285 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 16:57:52.779285 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 16:57:52.779285 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 16:57:52.779285 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 16:57:52.785501 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:57:52.785501 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:57:52.785501 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 16:57:52.785501 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:57:52.785501 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 16:57:52.785501 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:57:52.785501 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:57:52.785501 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:57:52.785501 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:57:52.785501 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:57:52.785501 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:57:52.785501 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:57:52.785501 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:57:52.785501 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:57:52.785501 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:57:52.785501 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:57:52.785501 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:57:52.785501 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:57:52.786440 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:57:52.786440 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:57:52.786440 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 16:57:52.786440 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 16:57:52.786440 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 16:57:52.786440 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 16:57:52.786440 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 16:57:52.786440 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 16:57:52.786440 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 18 16:57:52.786440 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:57:52.786440 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 631/1267] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:57:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:57:52.787058 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b697ff23-f3fd-4dce-b9c1-515d54372d4c None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:52.885508 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b697ff23-f3fd-4dce-b9c1-515d54372d4c None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:52.918549 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d86dac9-520b-4797-a421-1a2e4e85159a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:52.919188 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d86dac9-520b-4797-a421-1a2e4e85159a None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:52.919455 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d86dac9-520b-4797-a421-1a2e4e85159a None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:52.919726 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d86dac9-520b-4797-a421-1a2e4e85159a None admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:52.920432 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d86dac9-520b-4797-a421-1a2e4e85159a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:57:52.920787 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d86dac9-520b-4797-a421-1a2e4e85159a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:52.922163 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d86dac9-520b-4797-a421-1a2e4e85159a None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:52.931350 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:57:52.939119 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 637/1268] 10.222.0.82 () {56 vars in 1143 bytes} [Thu Aug 18 16:57:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 16:57:52.945583 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d9f18a-1125-4c7b-b812-d17f86168a37 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:53.045775 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d9f18a-1125-4c7b-b812-d17f86168a37 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:53.080254 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328fb6e8-54d5-4724-a675-2c3733263ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:53.080993 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328fb6e8-54d5-4724-a675-2c3733263ac5 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:53.081944 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328fb6e8-54d5-4724-a675-2c3733263ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:53.082265 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328fb6e8-54d5-4724-a675-2c3733263ac5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:53.093684 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328fb6e8-54d5-4724-a675-2c3733263ac5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:53.095343 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328fb6e8-54d5-4724-a675-2c3733263ac5 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:53.109949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-328fb6e8-54d5-4724-a675-2c3733263ac5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:57:53.109949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:57:53.109949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:57:53.109949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:57:53.109949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:57:53.109949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:57:53.109949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:57:53.109949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:57:53.109949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:57:53.109949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:57:53.109949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:57:53.109949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:57:53.109949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:57:53.109949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:57:53.109949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:57:53.109949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:57:53.110890 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:57:53.110890 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:57:53.110890 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:57:53.110890 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:57:53.110890 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:57:53.110890 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:57:53.110890 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:57:53.110890 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:57:53.110890 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:57:53.110890 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:57:53.110890 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:57:53.110890 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:57:53.110890 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:57:53.110890 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:57:53.110890 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:57:53.110890 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:57:53.110890 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:57:53.112016 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:57:53.112016 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:57:53.112016 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:57:53.112515 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 632/1269] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:57:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:57:53.117363 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d86dac9-520b-4797-a421-1a2e4e85159a None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:53.227316 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d86dac9-520b-4797-a421-1a2e4e85159a None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:53.262836 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a326971c-7f3c-484a-91b5-3ce2dd3bfff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:53.263487 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a326971c-7f3c-484a-91b5-3ce2dd3bfff7 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:53.263766 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a326971c-7f3c-484a-91b5-3ce2dd3bfff7 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:53.264038 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a326971c-7f3c-484a-91b5-3ce2dd3bfff7 None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:53.264670 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a326971c-7f3c-484a-91b5-3ce2dd3bfff7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:57:53.264993 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a326971c-7f3c-484a-91b5-3ce2dd3bfff7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:53.266327 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a326971c-7f3c-484a-91b5-3ce2dd3bfff7 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:53.278506 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 638/1270] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:57:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:57:53.286821 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328fb6e8-54d5-4724-a675-2c3733263ac5 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:53.316721 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:57:53.386871 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328fb6e8-54d5-4724-a675-2c3733263ac5 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:53.419359 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4f5d87-e7fd-4234-bae3-0c97ae56433d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:53.419984 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4f5d87-e7fd-4234-bae3-0c97ae56433d None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:53.420264 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4f5d87-e7fd-4234-bae3-0c97ae56433d None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:53.420539 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4f5d87-e7fd-4234-bae3-0c97ae56433d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:53.427475 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4f5d87-e7fd-4234-bae3-0c97ae56433d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:53.428922 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4f5d87-e7fd-4234-bae3-0c97ae56433d None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:53.441128 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-5b4f5d87-e7fd-4234-bae3-0c97ae56433d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:57:53.441128 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:57:53.441128 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:57:53.441128 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:57:53.441128 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:57:53.441128 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:57:53.441128 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:57:53.441128 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:57:53.441128 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:57:53.441128 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:57:53.441128 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:57:53.441128 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:57:53.441128 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 16:57:53.441128 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 16:57:53.441128 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 16:57:53.442019 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 16:57:53.442019 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:57:53.442019 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:57:53.442019 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:57:53.442019 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:57:53.442019 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:57:53.442019 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:57:53.442019 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:57:53.442019 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:57:53.442019 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:57:53.442019 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:57:53.442019 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:57:53.442019 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:57:53.442019 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:57:53.442019 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:57:53.442019 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:57:53.442019 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:57:53.442959 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 16:57:53.442959 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 16:57:53.442959 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 16:57:53.442959 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 16:57:53.442959 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 16:57:53.442959 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 16:57:53.442959 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:57:53.442959 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:57:53.442959 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 633/1271] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 16:57:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 16:57:53.448168 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a326971c-7f3c-484a-91b5-3ce2dd3bfff7 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:53.507557 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:57:53.549088 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a326971c-7f3c-484a-91b5-3ce2dd3bfff7 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:53.582406 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c1b3ca-ab44-41cb-ac04-ede34c16b129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:53.583026 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1b3ca-ab44-41cb-ac04-ede34c16b129 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:53.583310 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1b3ca-ab44-41cb-ac04-ede34c16b129 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:53.583579 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1b3ca-ab44-41cb-ac04-ede34c16b129 None admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:53.584259 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c1b3ca-ab44-41cb-ac04-ede34c16b129 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:57:53.584559 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c1b3ca-ab44-41cb-ac04-ede34c16b129 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:53.586041 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c1b3ca-ab44-41cb-ac04-ede34c16b129 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:53.597861 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 639/1272] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 16:57:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:57:53.603997 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4f5d87-e7fd-4234-bae3-0c97ae56433d None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:53.695542 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:57:53.704156 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4f5d87-e7fd-4234-bae3-0c97ae56433d None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:53.736513 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c45c27-5e73-4e5c-8437-0d7b989a9835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:53.737756 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c45c27-5e73-4e5c-8437-0d7b989a9835 None admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:53.738036 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c45c27-5e73-4e5c-8437-0d7b989a9835 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:53.738389 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c45c27-5e73-4e5c-8437-0d7b989a9835 None admin] PATH_INFO: `/v3/projects/2293c0aab61f44849d0d48b214cde7ea/users/29653ba1a2a74232972ef7a9000518b0/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:53.740295 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c45c27-5e73-4e5c-8437-0d7b989a9835 None admin] RBAC: Authorizing `identity:create_grant(project_id=2293c0aab61f44849d0d48b214cde7ea, user_id=29653ba1a2a74232972ef7a9000518b0, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:53.741737 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c45c27-5e73-4e5c-8437-0d7b989a9835 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:53.836915 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 634/1273] 10.222.0.82 () {58 vars in 1439 bytes} [Thu Aug 18 16:57:53 2022] PUT /identity/v3/projects/2293c0aab61f44849d0d48b214cde7ea/users/29653ba1a2a74232972ef7a9000518b0/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 16:57:55.738473 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81502c2-e5fd-4100-a111-63cb5f898f6a None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:55.739044 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81502c2-e5fd-4100-a111-63cb5f898f6a None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:55.739147 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81502c2-e5fd-4100-a111-63cb5f898f6a None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:55.740477 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 640/1274] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:57:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:57:55.745757 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba237dfc-5b9a-4822-ab38-4205dc79bd55 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:55.746130 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba237dfc-5b9a-4822-ab38-4205dc79bd55 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:55.748690 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba237dfc-5b9a-4822-ab38-4205dc79bd55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:55.764409 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-ba237dfc-5b9a-4822-ab38-4205dc79bd55 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:57:55.765603 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-ba237dfc-5b9a-4822-ab38-4205dc79bd55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:57:55.835000 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 635/1275] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:57:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 16:57:56.101483 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81502c2-e5fd-4100-a111-63cb5f898f6a None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:56.203306 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81502c2-e5fd-4100-a111-63cb5f898f6a None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:56.220035 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:57:56.235927 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92535dee-ca96-40d1-bfb4-71642b2a5aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:56.236580 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92535dee-ca96-40d1-bfb4-71642b2a5aae None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:56.237348 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92535dee-ca96-40d1-bfb4-71642b2a5aae None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:56.237842 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92535dee-ca96-40d1-bfb4-71642b2a5aae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:56.247449 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92535dee-ca96-40d1-bfb4-71642b2a5aae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 16:57:56.252243 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-92535dee-ca96-40d1-bfb4-71642b2a5aae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 16:57:56.252243 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:57:56.252243 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:57:56.252243 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:57:56.252243 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:57:56.252243 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:57:56.252243 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:57:56.252243 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:57:56.252243 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:57:56.252243 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:57:56.252243 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:57:56.252243 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:57:56.252243 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 16:57:56.252243 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 16:57:56.252243 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 16:57:56.252243 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 16:57:56.253727 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 16:57:56.253727 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 16:57:56.253727 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 16:57:56.253727 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 16:57:56.253727 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:57:56.253727 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:57:56.253727 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:57:56.253727 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:57:56.253727 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:57:56.253727 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:57:56.253727 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:57:56.253727 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:57:56.253727 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:57:56.253727 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:57:56.253727 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:57:56.253727 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:57:56.254556 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:57:56.254556 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:57:56.254556 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:57:56.254556 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:57:56.254556 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 16:57:56.254556 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 16:57:56.254556 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 16:57:56.254556 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 16:57:56.254556 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 16:57:56.254556 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 16:57:56.254556 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 16:57:56.254556 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:57:56.254556 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 641/1276] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 16:57:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:57:56.260388 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba237dfc-5b9a-4822-ab38-4205dc79bd55 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:56.339909 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:57:56.358144 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba237dfc-5b9a-4822-ab38-4205dc79bd55 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:56.393549 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd8649e-7230-4ba3-a200-ab421ed8842e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:56.394150 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd8649e-7230-4ba3-a200-ab421ed8842e None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:56.394484 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd8649e-7230-4ba3-a200-ab421ed8842e None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:56.394759 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd8649e-7230-4ba3-a200-ab421ed8842e None admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:56.395446 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd8649e-7230-4ba3-a200-ab421ed8842e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:57:56.395748 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd8649e-7230-4ba3-a200-ab421ed8842e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:56.397120 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd8649e-7230-4ba3-a200-ab421ed8842e None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:56.405286 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 636/1277] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 16:57:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:57:56.412175 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92535dee-ca96-40d1-bfb4-71642b2a5aae None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:56.512041 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92535dee-ca96-40d1-bfb4-71642b2a5aae None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:56.545354 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14057051-557c-44b2-b7d1-179721380b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:56.545979 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14057051-557c-44b2-b7d1-179721380b3c None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:56.546254 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14057051-557c-44b2-b7d1-179721380b3c None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:56.546524 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14057051-557c-44b2-b7d1-179721380b3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:56.557089 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14057051-557c-44b2-b7d1-179721380b3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:56.558513 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14057051-557c-44b2-b7d1-179721380b3c None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:56.573272 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-14057051-557c-44b2-b7d1-179721380b3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:57:56.573272 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:57:56.573272 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:57:56.573272 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:57:56.573272 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:57:56.573272 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:57:56.573272 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:57:56.573272 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:57:56.573272 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:57:56.573272 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:57:56.573272 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:57:56.573272 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:57:56.573272 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:57:56.573272 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:57:56.573272 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:57:56.573272 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:57:56.574406 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:57:56.574406 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:57:56.574406 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:57:56.574406 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:57:56.574406 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:57:56.574406 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:57:56.574406 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:57:56.574406 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:57:56.574406 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:57:56.574406 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:57:56.574406 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:57:56.574406 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:57:56.574406 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:57:56.574406 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:57:56.574406 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:57:56.574406 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:57:56.574406 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:57:56.575634 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:57:56.575634 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:57:56.575634 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:57:56.575634 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 642/1278] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:57:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:57:56.581056 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd8649e-7230-4ba3-a200-ab421ed8842e None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:56.679821 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd8649e-7230-4ba3-a200-ab421ed8842e None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:56.712555 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546a0b38-af5c-412d-9ad3-5317622bb82e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:56.713151 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546a0b38-af5c-412d-9ad3-5317622bb82e None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:56.713429 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546a0b38-af5c-412d-9ad3-5317622bb82e None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:56.713698 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546a0b38-af5c-412d-9ad3-5317622bb82e None admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:56.714323 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546a0b38-af5c-412d-9ad3-5317622bb82e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:57:56.714622 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546a0b38-af5c-412d-9ad3-5317622bb82e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:56.716022 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546a0b38-af5c-412d-9ad3-5317622bb82e None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:56.728067 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 637/1279] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:57:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:57:56.733933 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14057051-557c-44b2-b7d1-179721380b3c None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:56.765247 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:57:56.834530 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14057051-557c-44b2-b7d1-179721380b3c None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:56.869539 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9477a4-8841-4493-afb0-68daf51964fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:56.870248 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9477a4-8841-4493-afb0-68daf51964fa None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:56.870603 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9477a4-8841-4493-afb0-68daf51964fa None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:56.870969 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9477a4-8841-4493-afb0-68daf51964fa None admin] PATH_INFO: `/v3/users/placement` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:56.878604 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9477a4-8841-4493-afb0-68daf51964fa None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:56.880075 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9477a4-8841-4493-afb0-68daf51964fa None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:56.891294 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-0e9477a4-8841-4493-afb0-68daf51964fa None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 18 16:57:56.891294 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:57:56.891294 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:57:56.891294 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:57:56.891294 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:57:56.891294 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:57:56.891294 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:57:56.891294 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:57:56.891294 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:57:56.891294 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:57:56.891294 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:57:56.891294 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:57:56.891294 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 16:57:56.891294 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 16:57:56.891294 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 16:57:56.891294 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 16:57:56.896558 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:57:56.896558 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:57:56.896558 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 16:57:56.896558 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:57:56.896558 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 16:57:56.896558 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 16:57:56.896558 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:57:56.896558 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:57:56.896558 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:57:56.896558 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:57:56.896558 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:57:56.896558 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:57:56.896558 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:57:56.896558 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:57:56.896558 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:57:56.896558 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:57:56.896558 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:57:56.896558 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:57:56.897710 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:57:56.897710 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:57:56.897710 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 16:57:56.897710 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 16:57:56.897710 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 16:57:56.897710 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 16:57:56.897710 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 16:57:56.897710 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 16:57:56.897710 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 18 16:57:56.897710 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:57:56.897710 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 643/1280] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:57:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:57:56.899678 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546a0b38-af5c-412d-9ad3-5317622bb82e None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:56.903513 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:57:56.999143 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546a0b38-af5c-412d-9ad3-5317622bb82e None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:57.032797 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d171f6b-13e3-40b6-9382-83d5dd33ddc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:57.033391 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d171f6b-13e3-40b6-9382-83d5dd33ddc0 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:57.033885 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d171f6b-13e3-40b6-9382-83d5dd33ddc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:57.034165 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d171f6b-13e3-40b6-9382-83d5dd33ddc0 None admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:57.034883 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d171f6b-13e3-40b6-9382-83d5dd33ddc0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:57:57.035328 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d171f6b-13e3-40b6-9382-83d5dd33ddc0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:57.036758 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d171f6b-13e3-40b6-9382-83d5dd33ddc0 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:57.054793 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 638/1281] 10.222.0.82 () {56 vars in 1143 bytes} [Thu Aug 18 16:57:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:57:57.060520 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9477a4-8841-4493-afb0-68daf51964fa None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:57.150890 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:57:57.160707 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9477a4-8841-4493-afb0-68daf51964fa None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:57.193123 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65bfa59-7837-4eb5-a21b-39fae6ed1e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:57.193718 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65bfa59-7837-4eb5-a21b-39fae6ed1e81 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:57.193993 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65bfa59-7837-4eb5-a21b-39fae6ed1e81 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:57.194367 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65bfa59-7837-4eb5-a21b-39fae6ed1e81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:57.260361 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65bfa59-7837-4eb5-a21b-39fae6ed1e81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:57.261733 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65bfa59-7837-4eb5-a21b-39fae6ed1e81 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:57.348853 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:57:57.358577 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 644/1282] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 16:57:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 16:57:57.365222 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d171f6b-13e3-40b6-9382-83d5dd33ddc0 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:57.426070 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:57:57.463593 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d171f6b-13e3-40b6-9382-83d5dd33ddc0 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:57.498294 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba20bb80-2fce-4d0d-b49b-717a6e6d5a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:57.499011 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba20bb80-2fce-4d0d-b49b-717a6e6d5a89 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:57.499355 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba20bb80-2fce-4d0d-b49b-717a6e6d5a89 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:57.499574 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba20bb80-2fce-4d0d-b49b-717a6e6d5a89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:57.509379 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba20bb80-2fce-4d0d-b49b-717a6e6d5a89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:57.510948 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba20bb80-2fce-4d0d-b49b-717a6e6d5a89 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:57.525641 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-ba20bb80-2fce-4d0d-b49b-717a6e6d5a89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:57:57.525641 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:57:57.525641 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:57:57.525641 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:57:57.525641 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:57:57.525641 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:57:57.525641 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:57:57.525641 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:57:57.525641 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:57:57.525641 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:57:57.525641 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:57:57.525641 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:57:57.525641 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 16:57:57.525641 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 16:57:57.525641 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 16:57:57.525641 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 16:57:57.526987 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:57:57.526987 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:57:57.526987 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:57:57.526987 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:57:57.526987 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:57:57.526987 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:57:57.526987 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:57:57.526987 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:57:57.526987 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:57:57.526987 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:57:57.526987 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:57:57.526987 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:57:57.526987 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:57:57.526987 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:57:57.526987 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:57:57.526987 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:57:57.526987 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 16:57:57.527943 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 16:57:57.527943 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 16:57:57.527943 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:57:57.527943 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 639/1283] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 16:57:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:57:57.532763 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65bfa59-7837-4eb5-a21b-39fae6ed1e81 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:57.629384 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65bfa59-7837-4eb5-a21b-39fae6ed1e81 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:57.663411 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e195a3-eb74-4919-af7b-f2156721e713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:57.664006 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e195a3-eb74-4919-af7b-f2156721e713 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:57.664285 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e195a3-eb74-4919-af7b-f2156721e713 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:57.664553 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e195a3-eb74-4919-af7b-f2156721e713 None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:57.665180 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e195a3-eb74-4919-af7b-f2156721e713 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:57:57.665478 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e195a3-eb74-4919-af7b-f2156721e713 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:57.666825 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e195a3-eb74-4919-af7b-f2156721e713 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:57.677775 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 645/1284] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 16:57:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:57:57.683614 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba20bb80-2fce-4d0d-b49b-717a6e6d5a89 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:57.783218 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba20bb80-2fce-4d0d-b49b-717a6e6d5a89 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:57.797184 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:57:57.816618 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12fa191-5646-43ae-85dd-3aab4f06791a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:57.817249 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12fa191-5646-43ae-85dd-3aab4f06791a None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:57.817530 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12fa191-5646-43ae-85dd-3aab4f06791a None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:57.817801 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12fa191-5646-43ae-85dd-3aab4f06791a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:57.824303 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12fa191-5646-43ae-85dd-3aab4f06791a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:57.826003 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12fa191-5646-43ae-85dd-3aab4f06791a None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:57.837262 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application [None req-c12fa191-5646-43ae-85dd-3aab4f06791a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:57:57.837262 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:57:57.837262 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:57:57.837262 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:57:57.837262 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:57:57.837262 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:57:57.837262 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:57:57.837262 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:57:57.837262 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:57:57.837262 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:57:57.837262 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:57:57.837262 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:57:57.837262 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 16:57:57.837262 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 16:57:57.837262 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 16:57:57.838287 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 16:57:57.838287 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:57:57.838287 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:57:57.838287 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:57:57.838287 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:57:57.838287 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:57:57.838287 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:57:57.838287 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:57:57.838287 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application with Lock( Aug 18 16:57:57.838287 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:57:57.838287 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:57:57.838287 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:57:57.838287 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:57:57.838287 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:57:57.838287 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:57:57.838287 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:57:57.838287 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:57:57.839420 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 16:57:57.839420 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 16:57:57.839420 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 16:57:57.839420 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 16:57:57.839420 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 16:57:57.839420 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 16:57:57.839420 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 16:57:57.839420 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: ERROR keystone.server.flask.application Aug 18 16:57:57.839420 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 640/1285] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 16:57:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:57:57.844322 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e195a3-eb74-4919-af7b-f2156721e713 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:57.895290 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:57:57.943367 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e195a3-eb74-4919-af7b-f2156721e713 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:57.976777 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b410e29d-ab76-4304-9d3e-d2800993e3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:57.977373 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b410e29d-ab76-4304-9d3e-d2800993e3c7 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:57.977694 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b410e29d-ab76-4304-9d3e-d2800993e3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:57.977967 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b410e29d-ab76-4304-9d3e-d2800993e3c7 None admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:57.978642 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b410e29d-ab76-4304-9d3e-d2800993e3c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:57:57.978971 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b410e29d-ab76-4304-9d3e-d2800993e3c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:57.980395 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b410e29d-ab76-4304-9d3e-d2800993e3c7 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:57.992064 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 646/1286] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 16:57:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:57:57.998295 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12fa191-5646-43ae-85dd-3aab4f06791a None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:57:58.098382 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12fa191-5646-43ae-85dd-3aab4f06791a None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:57:58.131941 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7687854-f0dc-4e49-b42f-7f471c8e6787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:57:58.132539 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7687854-f0dc-4e49-b42f-7f471c8e6787 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:57:58.132817 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7687854-f0dc-4e49-b42f-7f471c8e6787 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:57:58.133085 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7687854-f0dc-4e49-b42f-7f471c8e6787 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:57:58.133794 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7687854-f0dc-4e49-b42f-7f471c8e6787 None admin] RBAC: Adding query filter params (role.id=2c6966cfb10f4d72b0242e9fb5e13b17, scope.project.id=2293c0aab61f44849d0d48b214cde7ea, user.id=29653ba1a2a74232972ef7a9000518b0) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:57:58.134097 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7687854-f0dc-4e49-b42f-7f471c8e6787 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:57:58.135560 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7687854-f0dc-4e49-b42f-7f471c8e6787 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:57:58.141096 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 641/1287] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 16:57:57 2022] GET /identity/v3/role_assignments?user.id=29653ba1a2a74232972ef7a9000518b0&scope.project.id=2293c0aab61f44849d0d48b214cde7ea&role.id=2c6966cfb10f4d72b0242e9fb5e13b17 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:58:00.054510 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f02f1-bea5-42f0-93d0-803bd3a4dce3 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:58:00.055110 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f02f1-bea5-42f0-93d0-803bd3a4dce3 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:58:00.055110 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f02f1-bea5-42f0-93d0-803bd3a4dce3 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:58:00.056484 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 647/1288] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:58:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:58:00.062223 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129a4b1d-e01e-472c-b3c3-fd7b5b5d9428 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:58:00.062730 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129a4b1d-e01e-472c-b3c3-fd7b5b5d9428 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:58:00.063026 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129a4b1d-e01e-472c-b3c3-fd7b5b5d9428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:58:00.083162 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-129a4b1d-e01e-472c-b3c3-fd7b5b5d9428 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:58:00.084371 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-129a4b1d-e01e-472c-b3c3-fd7b5b5d9428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:58:00.117198 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:58:00.157017 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 642/1289] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:58:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 16:58:00.431165 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1f02f1-bea5-42f0-93d0-803bd3a4dce3 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:58:00.531196 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1f02f1-bea5-42f0-93d0-803bd3a4dce3 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:58:00.542998 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:58:00.565890 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2750282f-4c9c-4e29-83c9-12f6e703c8b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:58:00.566516 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2750282f-4c9c-4e29-83c9-12f6e703c8b1 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:58:00.566796 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2750282f-4c9c-4e29-83c9-12f6e703c8b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:58:00.567101 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2750282f-4c9c-4e29-83c9-12f6e703c8b1 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:58:00.567680 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2750282f-4c9c-4e29-83c9-12f6e703c8b1 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:58:00.569046 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2750282f-4c9c-4e29-83c9-12f6e703c8b1 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:58:00.577820 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-2750282f-4c9c-4e29-83c9-12f6e703c8b1 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 18 16:58:00.577820 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:58:00.577820 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 16:58:00.577820 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 16:58:00.577820 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 16:58:00.577820 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 16:58:00.577820 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 16:58:00.577820 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 16:58:00.577820 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 18 16:58:00.577820 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:58:00.577820 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 16:58:00.577820 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:58:00.577820 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:58:00.577820 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:58:00.577820 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:58:00.577820 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:58:00.577820 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:58:00.579101 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:58:00.579101 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:58:00.579101 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:58:00.579101 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:58:00.579101 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:58:00.579101 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:58:00.579101 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 16:58:00.579101 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 16:58:00.579101 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 16:58:00.579101 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 16:58:00.579101 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:58:00.579101 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:58:00.579101 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:58:00.579101 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:58:00.579101 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:58:00.579101 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:58:00.580669 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:58:00.580669 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:58:00.580669 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:58:00.580669 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:58:00.580669 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:58:00.580669 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:58:00.580669 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:58:00.580669 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:58:00.580669 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:58:00.580669 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:58:00.580669 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 16:58:00.580669 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 16:58:00.580669 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 18 16:58:00.580669 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:58:00.580669 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 648/1290] 10.222.0.82 () {56 vars in 1117 bytes} [Thu Aug 18 16:58:00 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:58:00.585383 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129a4b1d-e01e-472c-b3c3-fd7b5b5d9428 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:58:00.681367 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129a4b1d-e01e-472c-b3c3-fd7b5b5d9428 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:58:00.715233 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd21a58-5ead-4ee9-b8b7-07bc347382da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:58:00.715836 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd21a58-5ead-4ee9-b8b7-07bc347382da None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:58:00.716117 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd21a58-5ead-4ee9-b8b7-07bc347382da None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:58:00.716389 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd21a58-5ead-4ee9-b8b7-07bc347382da None admin] PATH_INFO: `/v3/services` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:58:00.717019 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd21a58-5ead-4ee9-b8b7-07bc347382da None admin] RBAC: Adding query filter params (name=placement) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:58:00.717314 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd21a58-5ead-4ee9-b8b7-07bc347382da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:58:00.718681 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd21a58-5ead-4ee9-b8b7-07bc347382da None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:58:00.724147 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 643/1291] 10.222.0.82 () {56 vars in 1116 bytes} [Thu Aug 18 16:58:00 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:58:00.729921 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2750282f-4c9c-4e29-83c9-12f6e703c8b1 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:58:00.829851 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2750282f-4c9c-4e29-83c9-12f6e703c8b1 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:58:00.864395 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c59494-d1e4-47ed-ae82-1601cca15818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:58:00.864994 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c59494-d1e4-47ed-ae82-1601cca15818 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:58:00.865275 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c59494-d1e4-47ed-ae82-1601cca15818 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:58:00.865548 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c59494-d1e4-47ed-ae82-1601cca15818 None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:58:00.866166 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c59494-d1e4-47ed-ae82-1601cca15818 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:58:00.866468 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c59494-d1e4-47ed-ae82-1601cca15818 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:58:00.867857 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c59494-d1e4-47ed-ae82-1601cca15818 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:58:00.872200 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 649/1292] 10.222.0.82 () {56 vars in 1116 bytes} [Thu Aug 18 16:58:00 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:58:02.772706 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7605557-718d-4def-ba75-6a8e28009ffb None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:58:02.773122 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7605557-718d-4def-ba75-6a8e28009ffb None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:58:02.773228 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7605557-718d-4def-ba75-6a8e28009ffb None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:58:02.774539 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 644/1293] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:58:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:58:02.779600 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb8ee3-2a7b-419e-82e1-990ff8b2bffe None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:58:02.779976 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb8ee3-2a7b-419e-82e1-990ff8b2bffe None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:58:02.780147 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb8ee3-2a7b-419e-82e1-990ff8b2bffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:58:02.798531 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-c2fb8ee3-2a7b-419e-82e1-990ff8b2bffe None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:58:02.800473 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-c2fb8ee3-2a7b-419e-82e1-990ff8b2bffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:58:02.830000 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:58:02.865442 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 650/1294] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:58:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 16:58:03.144007 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7605557-718d-4def-ba75-6a8e28009ffb None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:58:03.175911 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:58:03.245306 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7605557-718d-4def-ba75-6a8e28009ffb None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:58:03.277065 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7ab66b-9b3a-422e-afaf-d9a0d145d6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:58:03.277660 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ab66b-9b3a-422e-afaf-d9a0d145d6fe None admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:58:03.277939 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ab66b-9b3a-422e-afaf-d9a0d145d6fe None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:58:03.278209 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ab66b-9b3a-422e-afaf-d9a0d145d6fe None admin] PATH_INFO: `/v3/services` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:58:03.278925 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7ab66b-9b3a-422e-afaf-d9a0d145d6fe None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:58:03.280305 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7ab66b-9b3a-422e-afaf-d9a0d145d6fe None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:58:03.374138 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 645/1295] 10.222.0.82 () {60 vars in 1141 bytes} [Thu Aug 18 16:58:03 2022] POST /identity/v3/services => generated 246 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 16:58:05.298123 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade96d80-469e-4e93-a705-15f334515afb None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:58:05.299384 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade96d80-469e-4e93-a705-15f334515afb None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:58:05.299384 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade96d80-469e-4e93-a705-15f334515afb None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:58:05.300633 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 651/1296] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:58:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:58:05.305740 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9934386-acb7-4449-b23c-e68df09d1dea None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:58:05.306140 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9934386-acb7-4449-b23c-e68df09d1dea None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:58:05.306286 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9934386-acb7-4449-b23c-e68df09d1dea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:58:05.324698 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-f9934386-acb7-4449-b23c-e68df09d1dea None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:58:05.325872 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-f9934386-acb7-4449-b23c-e68df09d1dea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:58:05.396959 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 646/1297] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:58:05 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 16:58:05.666083 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade96d80-469e-4e93-a705-15f334515afb None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:58:05.772663 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade96d80-469e-4e93-a705-15f334515afb None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:58:05.809142 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4c0f3a-f294-42ad-8dc9-bc959f9da9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:58:05.809772 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c0f3a-f294-42ad-8dc9-bc959f9da9a3 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:58:05.810054 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c0f3a-f294-42ad-8dc9-bc959f9da9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:58:05.810329 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c0f3a-f294-42ad-8dc9-bc959f9da9a3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:58:05.810918 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4c0f3a-f294-42ad-8dc9-bc959f9da9a3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:58:05.812293 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4c0f3a-f294-42ad-8dc9-bc959f9da9a3 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:58:05.820933 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-db4c0f3a-f294-42ad-8dc9-bc959f9da9a3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 18 16:58:05.820933 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:58:05.820933 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 16:58:05.820933 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 16:58:05.820933 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 16:58:05.820933 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 16:58:05.820933 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 16:58:05.820933 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 16:58:05.820933 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 18 16:58:05.820933 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:58:05.820933 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 16:58:05.820933 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:58:05.820933 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:58:05.820933 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:58:05.820933 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:58:05.820933 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:58:05.820933 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:58:05.824502 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:58:05.824502 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:58:05.824502 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:58:05.824502 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:58:05.824502 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:58:05.824502 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:58:05.824502 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 16:58:05.824502 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 16:58:05.824502 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 16:58:05.824502 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 16:58:05.824502 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:58:05.824502 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:58:05.824502 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:58:05.824502 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:58:05.824502 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:58:05.824502 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:58:05.825335 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:58:05.825335 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:58:05.825335 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:58:05.825335 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:58:05.825335 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:58:05.825335 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:58:05.825335 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:58:05.825335 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:58:05.825335 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:58:05.825335 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:58:05.825335 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 16:58:05.825335 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 16:58:05.825335 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 18 16:58:05.825335 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:58:05.825335 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 652/1298] 10.222.0.82 () {56 vars in 1117 bytes} [Thu Aug 18 16:58:05 2022] GET /identity/v3/services/placement => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:58:05.828549 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9934386-acb7-4449-b23c-e68df09d1dea None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:58:05.847494 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:58:05.927058 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9934386-acb7-4449-b23c-e68df09d1dea None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:58:05.960284 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a2db70-7d2a-436a-b185-78c323f910a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:58:05.960888 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a2db70-7d2a-436a-b185-78c323f910a8 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:58:05.961168 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a2db70-7d2a-436a-b185-78c323f910a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:58:05.961442 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a2db70-7d2a-436a-b185-78c323f910a8 None admin] PATH_INFO: `/v3/services` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:58:05.962062 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a2db70-7d2a-436a-b185-78c323f910a8 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:58:05.962367 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a2db70-7d2a-436a-b185-78c323f910a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:58:05.963798 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a2db70-7d2a-436a-b185-78c323f910a8 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:58:05.969668 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 647/1299] 10.222.0.82 () {56 vars in 1116 bytes} [Thu Aug 18 16:58:05 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:58:05.975838 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4c0f3a-f294-42ad-8dc9-bc959f9da9a3 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:58:06.007333 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:58:06.073190 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4c0f3a-f294-42ad-8dc9-bc959f9da9a3 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:58:06.106384 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2092d65-526c-4953-bbf4-68eb38b36959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:58:06.107004 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2092d65-526c-4953-bbf4-68eb38b36959 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:58:06.107286 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2092d65-526c-4953-bbf4-68eb38b36959 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:58:06.107558 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2092d65-526c-4953-bbf4-68eb38b36959 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:58:06.108241 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2092d65-526c-4953-bbf4-68eb38b36959 None admin] RBAC: Adding query filter params (interface=public, service_id=c30a3e5adf174b60906c5d7194571f45, region_id=RegionOne) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:58:06.108548 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2092d65-526c-4953-bbf4-68eb38b36959 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:58:06.109895 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2092d65-526c-4953-bbf4-68eb38b36959 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:58:06.115327 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 653/1300] 10.222.0.82 () {56 vars in 1251 bytes} [Thu Aug 18 16:58:05 2022] GET /identity/v3/endpoints?service_id=c30a3e5adf174b60906c5d7194571f45&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:58:06.121071 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a2db70-7d2a-436a-b185-78c323f910a8 None admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:58:06.201605 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:58:06.218549 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a2db70-7d2a-436a-b185-78c323f910a8 None admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:58:06.251213 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56e3bb3-3ef8-4af9-b62f-1fb389267b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:58:06.251918 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56e3bb3-3ef8-4af9-b62f-1fb389267b80 None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:58:06.252200 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56e3bb3-3ef8-4af9-b62f-1fb389267b80 None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:58:06.252472 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56e3bb3-3ef8-4af9-b62f-1fb389267b80 None admin] PATH_INFO: `/v3/services` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:58:06.253062 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56e3bb3-3ef8-4af9-b62f-1fb389267b80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:58:06.254433 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56e3bb3-3ef8-4af9-b62f-1fb389267b80 None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:58:06.260926 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 648/1301] 10.222.0.82 () {56 vars in 1087 bytes} [Thu Aug 18 16:58:06 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 16:58:08.153334 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54da7d5-d313-493e-9bc8-0fe52d30d47d None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:58:08.153903 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54da7d5-d313-493e-9bc8-0fe52d30d47d None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:58:08.153903 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54da7d5-d313-493e-9bc8-0fe52d30d47d None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:58:08.155271 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 654/1302] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 16:58:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 16:58:08.160701 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abb21bd-0820-4d97-bc4c-cded5165c51a None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:58:08.161141 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abb21bd-0820-4d97-bc4c-cded5165c51a None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:58:08.161269 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abb21bd-0820-4d97-bc4c-cded5165c51a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:58:08.179626 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-6abb21bd-0820-4d97-bc4c-cded5165c51a None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 16:58:08.180821 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-6abb21bd-0820-4d97-bc4c-cded5165c51a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 16:58:08.247007 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 649/1303] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 16:58:08 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 16:58:08.515329 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54da7d5-d313-493e-9bc8-0fe52d30d47d None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:58:08.609314 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:58:08.616711 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54da7d5-d313-493e-9bc8-0fe52d30d47d None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:58:08.648674 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f3dfc-8487-425b-85db-339ee36702bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:58:08.649292 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f3dfc-8487-425b-85db-339ee36702bb None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:58:08.649571 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f3dfc-8487-425b-85db-339ee36702bb None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:58:08.649882 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f3dfc-8487-425b-85db-339ee36702bb None admin] PATH_INFO: `/v3/services/placement` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:58:08.650452 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37f3dfc-8487-425b-85db-339ee36702bb None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:58:08.651907 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37f3dfc-8487-425b-85db-339ee36702bb None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:58:08.660908 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-a37f3dfc-8487-425b-85db-339ee36702bb None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 18 16:58:08.660908 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:58:08.660908 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 16:58:08.660908 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 16:58:08.660908 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 16:58:08.660908 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 16:58:08.660908 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 16:58:08.660908 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 16:58:08.660908 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 18 16:58:08.660908 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:58:08.660908 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 16:58:08.660908 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:58:08.660908 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 16:58:08.660908 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 16:58:08.660908 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 16:58:08.660908 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 16:58:08.660908 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 16:58:08.661970 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 16:58:08.661970 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 16:58:08.661970 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 16:58:08.661970 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 16:58:08.661970 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 16:58:08.661970 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 16:58:08.661970 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 16:58:08.661970 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 16:58:08.661970 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 16:58:08.661970 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 16:58:08.661970 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 16:58:08.661970 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 16:58:08.661970 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 16:58:08.661970 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 16:58:08.661970 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 16:58:08.661970 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 16:58:08.663289 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 16:58:08.663289 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Aug 18 16:58:08.663289 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 16:58:08.663289 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Aug 18 16:58:08.663289 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 16:58:08.663289 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 16:58:08.663289 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 16:58:08.663289 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Aug 18 16:58:08.663289 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 16:58:08.663289 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Aug 18 16:58:08.663289 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 16:58:08.663289 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 16:58:08.663289 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 18 16:58:08.663289 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Aug 18 16:58:08.663289 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 655/1304] 10.222.0.82 () {56 vars in 1117 bytes} [Thu Aug 18 16:58:08 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 16:58:08.668064 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abb21bd-0820-4d97-bc4c-cded5165c51a None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:58:08.744667 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:58:08.766501 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abb21bd-0820-4d97-bc4c-cded5165c51a None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:58:08.800890 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd6c77b-311a-4b34-9ee3-0060e9d78d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:58:08.801502 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd6c77b-311a-4b34-9ee3-0060e9d78d2f None admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:58:08.801784 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd6c77b-311a-4b34-9ee3-0060e9d78d2f None admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:58:08.802056 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd6c77b-311a-4b34-9ee3-0060e9d78d2f None admin] PATH_INFO: `/v3/services` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:58:08.802674 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd6c77b-311a-4b34-9ee3-0060e9d78d2f None admin] RBAC: Adding query filter params (name=placement) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 16:58:08.803004 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd6c77b-311a-4b34-9ee3-0060e9d78d2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:58:08.804444 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd6c77b-311a-4b34-9ee3-0060e9d78d2f None admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:58:08.809320 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 650/1305] 10.222.0.82 () {56 vars in 1116 bytes} [Thu Aug 18 16:58:08 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 16:58:08.815245 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f3dfc-8487-425b-85db-339ee36702bb None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 16:58:08.915709 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f3dfc-8487-425b-85db-339ee36702bb None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 16:58:08.949195 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0722aed-02bf-47ca-9940-9595eed02253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 16:58:08.949793 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0722aed-02bf-47ca-9940-9595eed02253 None admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 16:58:08.950074 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0722aed-02bf-47ca-9940-9595eed02253 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 16:58:08.950349 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0722aed-02bf-47ca-9940-9595eed02253 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 16:58:08.951018 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0722aed-02bf-47ca-9940-9595eed02253 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 16:58:08.952371 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0722aed-02bf-47ca-9940-9595eed02253 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 16:58:09.028230 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 656/1306] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 16:58:08 2022] POST /identity/v3/endpoints => generated 331 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 16:58:18.857212 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=30 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 16:58:19.023869 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:04:12.106860 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5030ae3-6049-4a00-93bb-8873e184dbf4 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:04:12.107331 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5030ae3-6049-4a00-93bb-8873e184dbf4 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:04:12.107422 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5030ae3-6049-4a00-93bb-8873e184dbf4 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:04:12.115123 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e49fd-d9ac-4354-aa30-0e3b6f14da71 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:04:12.121980 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e49fd-d9ac-4354-aa30-0e3b6f14da71 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:04:12.121980 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e49fd-d9ac-4354-aa30-0e3b6f14da71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:04:12.122665 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 651/1307] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 17:04:12 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:04:12.216171 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-8f8e49fd-d9ac-4354-aa30-0e3b6f14da71 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:04:12.217958 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-8f8e49fd-d9ac-4354-aa30-0e3b6f14da71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:04:12.343515 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 657/1308] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 17:04:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:04:12.680365 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13527d0e-d166-4a5f-8ec7-2c2d0943153b None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:04:12.680923 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13527d0e-d166-4a5f-8ec7-2c2d0943153b None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:04:12.681340 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13527d0e-d166-4a5f-8ec7-2c2d0943153b None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:04:12.683162 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 652/1309] 10.222.0.82 () {54 vars in 965 bytes} [Thu Aug 18 17:04:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:04:12.689601 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969971d5-bcb6-4d34-b1d3-9f234a535dd1 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:04:12.689896 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969971d5-bcb6-4d34-b1d3-9f234a535dd1 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:04:12.690178 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969971d5-bcb6-4d34-b1d3-9f234a535dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:04:12.778555 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-969971d5-bcb6-4d34-b1d3-9f234a535dd1 None None] MFA Rules not processed for user `fe2403ede53148b2a113ffa2a605e987`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:04:12.785631 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-969971d5-bcb6-4d34-b1d3-9f234a535dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:04:12.809513 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:04:12.888894 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 658/1310] 10.222.0.82 () {58 vars in 1062 bytes} [Thu Aug 18 17:04:12 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:04:12.897994 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13527d0e-d166-4a5f-8ec7-2c2d0943153b None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:04:12.900247 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-13527d0e-d166-4a5f-8ec7-2c2d0943153b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:04:13.033478 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13527d0e-d166-4a5f-8ec7-2c2d0943153b None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:04:13.119867 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec6c306-fc0e-4946-b55c-c405efaab844 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:04:13.121453 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec6c306-fc0e-4946-b55c-c405efaab844 service neutron] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:04:13.122073 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec6c306-fc0e-4946-b55c-c405efaab844 service neutron] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:04:13.122582 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec6c306-fc0e-4946-b55c-c405efaab844 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:04:13.124924 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-5ec6c306-fc0e-4946-b55c-c405efaab844 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:04:13.188572 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:04:13.253140 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec6c306-fc0e-4946-b55c-c405efaab844 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:04:13.254960 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec6c306-fc0e-4946-b55c-c405efaab844 service neutron] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:04:13.279676 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 653/1311] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:04:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:04:15.536640 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd997fd7-0346-46fd-a313-04ce0eb43312 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:04:15.537210 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd997fd7-0346-46fd-a313-04ce0eb43312 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:04:15.537210 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd997fd7-0346-46fd-a313-04ce0eb43312 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:04:15.538545 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 659/1312] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 17:04:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:04:15.544294 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b4991-f554-438a-aa8a-47e59158ced0 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:04:15.544814 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b4991-f554-438a-aa8a-47e59158ced0 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:04:15.544814 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b4991-f554-438a-aa8a-47e59158ced0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:04:15.618917 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-776b4991-f554-438a-aa8a-47e59158ced0 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:04:15.620118 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-776b4991-f554-438a-aa8a-47e59158ced0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:04:15.716059 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 654/1313] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 17:04:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:04:16.006957 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd997fd7-0346-46fd-a313-04ce0eb43312 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:04:16.008622 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-dd997fd7-0346-46fd-a313-04ce0eb43312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:04:16.110733 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd997fd7-0346-46fd-a313-04ce0eb43312 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:04:16.179894 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:04:16.182376 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f6f669-623a-42ae-8679-610c9cd02bbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:04:16.183006 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f6f669-623a-42ae-8679-610c9cd02bbb service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:04:16.183283 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f6f669-623a-42ae-8679-610c9cd02bbb service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:04:16.183554 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f6f669-623a-42ae-8679-610c9cd02bbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:04:16.185123 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-17f6f669-623a-42ae-8679-610c9cd02bbb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:04:16.290840 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f6f669-623a-42ae-8679-610c9cd02bbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:04:16.292294 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f6f669-623a-42ae-8679-610c9cd02bbb service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:04:16.310696 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 660/1314] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:04:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:04:18.637837 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b373c57-06b8-47d7-aaad-aed19efc00cc None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:04:18.638372 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b373c57-06b8-47d7-aaad-aed19efc00cc None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:04:18.638372 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b373c57-06b8-47d7-aaad-aed19efc00cc None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:04:18.639750 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 655/1315] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 17:04:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:04:18.645384 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d03f85-0837-4992-9051-d36573d35df1 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:04:18.645772 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d03f85-0837-4992-9051-d36573d35df1 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:04:18.645965 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d03f85-0837-4992-9051-d36573d35df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:04:18.710789 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-10d03f85-0837-4992-9051-d36573d35df1 None None] MFA Rules not processed for user `3f70ad03afa246c393b5448fe7ee6860`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:04:18.712016 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-10d03f85-0837-4992-9051-d36573d35df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:04:18.761771 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:04:18.806368 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 661/1316] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 17:04:18 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:04:19.094844 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b373c57-06b8-47d7-aaad-aed19efc00cc None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:04:19.096440 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-5b373c57-06b8-47d7-aaad-aed19efc00cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:04:19.106232 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:04:19.197460 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b373c57-06b8-47d7-aaad-aed19efc00cc None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:04:19.268587 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f323b834-e100-4516-9c0e-ab07e065de53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:04:19.269191 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f323b834-e100-4516-9c0e-ab07e065de53 service neutron] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:04:19.269469 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f323b834-e100-4516-9c0e-ab07e065de53 service neutron] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:04:19.269738 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f323b834-e100-4516-9c0e-ab07e065de53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:04:19.271345 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-f323b834-e100-4516-9c0e-ab07e065de53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:04:19.333277 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:04:19.380013 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f323b834-e100-4516-9c0e-ab07e065de53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:04:19.381674 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f323b834-e100-4516-9c0e-ab07e065de53 service neutron] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:04:19.400143 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 656/1317] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:04:19 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:04:22.155717 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74cd85d-b286-42db-94a7-f66ab54bf588 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:04:22.156310 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74cd85d-b286-42db-94a7-f66ab54bf588 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:04:22.156310 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74cd85d-b286-42db-94a7-f66ab54bf588 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:04:22.157611 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 662/1318] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 17:04:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:04:22.162765 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ee6e6-8c32-49ef-96f3-13f8837adb4e None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:04:22.163228 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ee6e6-8c32-49ef-96f3-13f8837adb4e None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:04:22.163368 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ee6e6-8c32-49ef-96f3-13f8837adb4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:04:22.231647 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-914ee6e6-8c32-49ef-96f3-13f8837adb4e None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:04:22.232858 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-914ee6e6-8c32-49ef-96f3-13f8837adb4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:04:22.328125 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 657/1319] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 17:04:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:04:22.615516 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74cd85d-b286-42db-94a7-f66ab54bf588 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:04:22.617051 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-c74cd85d-b286-42db-94a7-f66ab54bf588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:04:22.720114 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74cd85d-b286-42db-94a7-f66ab54bf588 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:04:22.791532 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b61ef6-406c-4cd8-a4e7-36a0c853ab8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:04:22.792138 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b61ef6-406c-4cd8-a4e7-36a0c853ab8c service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:04:22.792414 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b61ef6-406c-4cd8-a4e7-36a0c853ab8c service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:04:22.792680 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b61ef6-406c-4cd8-a4e7-36a0c853ab8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:04:22.794258 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-b7b61ef6-406c-4cd8-a4e7-36a0c853ab8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:04:22.802163 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:04:22.900403 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b61ef6-406c-4cd8-a4e7-36a0c853ab8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:04:22.901795 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b61ef6-406c-4cd8-a4e7-36a0c853ab8c service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:04:22.919764 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 663/1320] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:04:22 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:04:25.834074 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f7c499-1d50-4a1e-b686-dee24789f109 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:04:25.834498 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f7c499-1d50-4a1e-b686-dee24789f109 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:04:25.834625 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f7c499-1d50-4a1e-b686-dee24789f109 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:04:25.836149 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 658/1321] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 17:04:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:04:25.841069 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e323f29-b8cc-4745-9c2d-c95f3d85716c None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:04:25.841467 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e323f29-b8cc-4745-9c2d-c95f3d85716c None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:04:25.841642 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e323f29-b8cc-4745-9c2d-c95f3d85716c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:04:25.908348 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-5e323f29-b8cc-4745-9c2d-c95f3d85716c None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:04:25.909560 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-5e323f29-b8cc-4745-9c2d-c95f3d85716c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:04:25.967521 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:04:26.006714 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 664/1322] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 17:04:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:04:26.316220 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478aa143-8491-4782-9b09-51104bec6554 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:04:26.316596 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478aa143-8491-4782-9b09-51104bec6554 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:04:26.316944 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478aa143-8491-4782-9b09-51104bec6554 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:04:26.318345 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 659/1323] 10.222.0.82 () {54 vars in 965 bytes} [Thu Aug 18 17:04:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:04:26.324784 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e663f4a2-59d1-4bd4-884c-24f7abc6cf8a None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:04:26.325133 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e663f4a2-59d1-4bd4-884c-24f7abc6cf8a None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:04:26.325467 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e663f4a2-59d1-4bd4-884c-24f7abc6cf8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:04:26.409467 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-e663f4a2-59d1-4bd4-884c-24f7abc6cf8a None None] MFA Rules not processed for user `fe2403ede53148b2a113ffa2a605e987`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:04:26.410667 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-e663f4a2-59d1-4bd4-884c-24f7abc6cf8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:04:26.486281 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 665/1324] 10.222.0.82 () {58 vars in 1062 bytes} [Thu Aug 18 17:04:26 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 17:04:26.495206 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478aa143-8491-4782-9b09-51104bec6554 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:04:26.496848 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-478aa143-8491-4782-9b09-51104bec6554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:04:26.518025 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:04:26.605153 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478aa143-8491-4782-9b09-51104bec6554 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:04:26.678844 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01424438-31fc-487d-a431-4e136e75a31c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:04:26.679482 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01424438-31fc-487d-a431-4e136e75a31c service neutron] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:04:26.679765 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01424438-31fc-487d-a431-4e136e75a31c service neutron] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:04:26.680040 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01424438-31fc-487d-a431-4e136e75a31c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:04:26.681631 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-01424438-31fc-487d-a431-4e136e75a31c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:04:26.752820 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:04:26.790664 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01424438-31fc-487d-a431-4e136e75a31c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:04:26.792152 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01424438-31fc-487d-a431-4e136e75a31c service neutron] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:04:26.809854 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 660/1325] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:04:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:04:29.056781 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f02a91-b233-4010-b1fe-d567519179cf None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:04:29.057445 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f02a91-b233-4010-b1fe-d567519179cf None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:04:29.057445 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f02a91-b233-4010-b1fe-d567519179cf None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:04:29.058780 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 666/1326] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 17:04:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:04:29.065584 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1e93da-6894-4402-a24d-a0a0495544de None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:04:29.066107 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1e93da-6894-4402-a24d-a0a0495544de None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:04:29.066253 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1e93da-6894-4402-a24d-a0a0495544de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:04:29.142501 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-ee1e93da-6894-4402-a24d-a0a0495544de None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:04:29.143793 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-ee1e93da-6894-4402-a24d-a0a0495544de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:04:29.248465 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 661/1327] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 17:04:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:04:29.532370 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f02a91-b233-4010-b1fe-d567519179cf None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:04:29.533983 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-36f02a91-b233-4010-b1fe-d567519179cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:04:29.595623 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:04:29.639443 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f02a91-b233-4010-b1fe-d567519179cf None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:04:29.711973 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7eb638-dd47-4b82-9873-54b4074da212 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:04:29.712640 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7eb638-dd47-4b82-9873-54b4074da212 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:04:29.712923 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7eb638-dd47-4b82-9873-54b4074da212 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:04:29.713194 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7eb638-dd47-4b82-9873-54b4074da212 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:04:29.714861 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-0b7eb638-dd47-4b82-9873-54b4074da212 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:04:29.824761 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:04:29.827626 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7eb638-dd47-4b82-9873-54b4074da212 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:04:29.829133 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7eb638-dd47-4b82-9873-54b4074da212 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:04:29.848224 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 667/1328] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:04:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:04:31.819654 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61d0d0-ad49-467b-9566-80e3586fa2d6 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:04:31.820125 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61d0d0-ad49-467b-9566-80e3586fa2d6 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:04:31.820224 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61d0d0-ad49-467b-9566-80e3586fa2d6 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:04:31.821524 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 662/1329] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 17:04:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:04:31.826821 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae85839d-504d-4eee-b24a-f8de48a6acbb None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:04:31.827840 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae85839d-504d-4eee-b24a-f8de48a6acbb None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:04:31.827976 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae85839d-504d-4eee-b24a-f8de48a6acbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:04:31.893641 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-ae85839d-504d-4eee-b24a-f8de48a6acbb None None] MFA Rules not processed for user `3f70ad03afa246c393b5448fe7ee6860`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:04:31.894893 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-ae85839d-504d-4eee-b24a-f8de48a6acbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:04:32.010630 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 668/1330] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 17:04:31 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:04:32.288784 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b61d0d0-ad49-467b-9566-80e3586fa2d6 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:04:32.290406 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-4b61d0d0-ad49-467b-9566-80e3586fa2d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:04:32.322500 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:04:32.395489 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b61d0d0-ad49-467b-9566-80e3586fa2d6 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:04:32.469421 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f81d41-1f07-47eb-a9d6-01a99bf55319 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:04:32.470028 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f81d41-1f07-47eb-a9d6-01a99bf55319 service neutron] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:04:32.470305 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f81d41-1f07-47eb-a9d6-01a99bf55319 service neutron] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:04:32.470572 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f81d41-1f07-47eb-a9d6-01a99bf55319 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:04:32.472178 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-36f81d41-1f07-47eb-a9d6-01a99bf55319 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:04:32.557386 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:04:32.592902 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f81d41-1f07-47eb-a9d6-01a99bf55319 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:04:32.594380 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f81d41-1f07-47eb-a9d6-01a99bf55319 service neutron] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:04:32.614050 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 663/1331] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:04:32 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:04:34.989870 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb577d6-6277-48ac-b775-3418886334a2 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:04:34.990395 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb577d6-6277-48ac-b775-3418886334a2 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:04:34.990395 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb577d6-6277-48ac-b775-3418886334a2 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:04:34.991805 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 669/1332] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 17:04:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:04:34.997018 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2204902-41c2-415b-a154-c7b139c1ed38 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:04:34.997475 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2204902-41c2-415b-a154-c7b139c1ed38 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:04:34.997573 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2204902-41c2-415b-a154-c7b139c1ed38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:04:35.063407 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-c2204902-41c2-415b-a154-c7b139c1ed38 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:04:35.064695 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-c2204902-41c2-415b-a154-c7b139c1ed38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:04:35.160250 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 664/1333] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 17:04:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:04:35.438909 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb577d6-6277-48ac-b775-3418886334a2 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:04:35.440480 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-fcb577d6-6277-48ac-b775-3418886334a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:04:35.509988 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:04:35.544787 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb577d6-6277-48ac-b775-3418886334a2 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:04:35.616905 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79acff9f-516a-4f87-a36c-557a8c8fd9cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:04:35.617505 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79acff9f-516a-4f87-a36c-557a8c8fd9cd service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:04:35.617782 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79acff9f-516a-4f87-a36c-557a8c8fd9cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:04:35.618051 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79acff9f-516a-4f87-a36c-557a8c8fd9cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:04:35.619661 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-79acff9f-516a-4f87-a36c-557a8c8fd9cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:04:35.726536 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79acff9f-516a-4f87-a36c-557a8c8fd9cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:04:35.727912 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79acff9f-516a-4f87-a36c-557a8c8fd9cd service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:04:35.738116 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:04:35.746052 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 670/1334] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:04:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:04:37.699124 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d51855f-b772-417e-9276-7cf8f1438c1f None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:04:37.699627 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d51855f-b772-417e-9276-7cf8f1438c1f None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:04:37.699627 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d51855f-b772-417e-9276-7cf8f1438c1f None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:04:37.700934 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 665/1335] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 17:04:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:04:37.706395 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f99ae79-2565-475c-a51e-f41556d79e60 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:04:37.706820 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f99ae79-2565-475c-a51e-f41556d79e60 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:04:37.706974 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f99ae79-2565-475c-a51e-f41556d79e60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:04:37.772253 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-0f99ae79-2565-475c-a51e-f41556d79e60 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:04:37.773465 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-0f99ae79-2565-475c-a51e-f41556d79e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:04:37.868484 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 671/1336] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 17:04:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:04:38.159037 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d51855f-b772-417e-9276-7cf8f1438c1f None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:04:38.160580 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-7d51855f-b772-417e-9276-7cf8f1438c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:04:38.200121 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:04:38.263101 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d51855f-b772-417e-9276-7cf8f1438c1f None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:04:38.336510 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68ff665-5346-4fce-b5fc-d7ffc4b91cc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:04:38.337106 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ff665-5346-4fce-b5fc-d7ffc4b91cc3 service neutron] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:04:38.337383 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ff665-5346-4fce-b5fc-d7ffc4b91cc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:04:38.337738 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ff665-5346-4fce-b5fc-d7ffc4b91cc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:04:38.339344 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-d68ff665-5346-4fce-b5fc-d7ffc4b91cc3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:04:38.422214 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:04:38.447286 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68ff665-5346-4fce-b5fc-d7ffc4b91cc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:04:38.448975 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68ff665-5346-4fce-b5fc-d7ffc4b91cc3 service neutron] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:04:38.467876 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 666/1337] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:04:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:04:41.028580 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc74066a-4887-4fec-ab58-05150b7d98b2 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:04:41.029086 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc74066a-4887-4fec-ab58-05150b7d98b2 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:04:41.029086 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc74066a-4887-4fec-ab58-05150b7d98b2 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:04:41.030461 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 672/1338] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 17:04:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:04:41.035970 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852fd43-7505-4e34-a3d9-45d6ffc7f673 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:04:41.036379 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852fd43-7505-4e34-a3d9-45d6ffc7f673 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:04:41.036525 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852fd43-7505-4e34-a3d9-45d6ffc7f673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:04:41.102921 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-0852fd43-7505-4e34-a3d9-45d6ffc7f673 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:04:41.104200 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-0852fd43-7505-4e34-a3d9-45d6ffc7f673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:04:41.200159 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 667/1339] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 17:04:41 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:04:41.485086 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc74066a-4887-4fec-ab58-05150b7d98b2 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:04:41.486649 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-dc74066a-4887-4fec-ab58-05150b7d98b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:04:41.562605 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:04:41.589979 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc74066a-4887-4fec-ab58-05150b7d98b2 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:04:41.663071 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494e13cc-c452-45bb-b3ac-6f51c991fe65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:04:41.663680 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e13cc-c452-45bb-b3ac-6f51c991fe65 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:04:41.663961 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e13cc-c452-45bb-b3ac-6f51c991fe65 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:04:41.664235 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e13cc-c452-45bb-b3ac-6f51c991fe65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:04:41.665833 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-494e13cc-c452-45bb-b3ac-6f51c991fe65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:04:41.773353 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494e13cc-c452-45bb-b3ac-6f51c991fe65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:04:41.774785 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494e13cc-c452-45bb-b3ac-6f51c991fe65 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:04:41.793125 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 673/1340] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:04:41 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:04:45.888275 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d728e30-1415-4063-a608-b05e82519cd6 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:04:45.888826 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d728e30-1415-4063-a608-b05e82519cd6 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:04:45.888932 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d728e30-1415-4063-a608-b05e82519cd6 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:04:45.890247 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 668/1341] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 17:04:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:04:45.895454 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a7207-ba2e-4732-8282-65626732862d None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:04:45.895883 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a7207-ba2e-4732-8282-65626732862d None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:04:45.896016 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a7207-ba2e-4732-8282-65626732862d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:04:45.901402 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:04:45.963732 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-784a7207-ba2e-4732-8282-65626732862d None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:04:45.964963 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-784a7207-ba2e-4732-8282-65626732862d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:04:46.064206 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 674/1342] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 17:04:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:04:46.347363 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d728e30-1415-4063-a608-b05e82519cd6 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:04:46.348923 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-7d728e30-1415-4063-a608-b05e82519cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:04:46.397306 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:04:46.455252 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d728e30-1415-4063-a608-b05e82519cd6 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:04:46.528765 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8d1ee7-8c87-4259-a1ba-5eeadf3ddc70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:04:46.529969 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d1ee7-8c87-4259-a1ba-5eeadf3ddc70 service neutron] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:04:46.530444 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d1ee7-8c87-4259-a1ba-5eeadf3ddc70 service neutron] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:04:46.530831 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d1ee7-8c87-4259-a1ba-5eeadf3ddc70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:04:46.532453 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-2f8d1ee7-8c87-4259-a1ba-5eeadf3ddc70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:04:46.628998 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:04:46.645064 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8d1ee7-8c87-4259-a1ba-5eeadf3ddc70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:04:46.646513 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8d1ee7-8c87-4259-a1ba-5eeadf3ddc70 service neutron] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:04:46.665655 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 669/1343] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:04:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:04:49.605934 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12612f75-e7ce-4b88-bee5-8004760a4df8 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:04:49.606439 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12612f75-e7ce-4b88-bee5-8004760a4df8 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:04:49.606439 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12612f75-e7ce-4b88-bee5-8004760a4df8 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:04:49.607858 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 675/1344] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 17:04:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:04:49.613044 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b872d580-13af-4b59-bda6-876b68ec6886 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:04:49.613489 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b872d580-13af-4b59-bda6-876b68ec6886 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:04:49.613716 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b872d580-13af-4b59-bda6-876b68ec6886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:04:49.679571 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-b872d580-13af-4b59-bda6-876b68ec6886 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:04:49.680785 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-b872d580-13af-4b59-bda6-876b68ec6886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:04:49.777544 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 670/1345] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 17:04:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:04:50.066344 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12612f75-e7ce-4b88-bee5-8004760a4df8 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:04:50.067930 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-12612f75-e7ce-4b88-bee5-8004760a4df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:04:50.158067 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:04:50.176020 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12612f75-e7ce-4b88-bee5-8004760a4df8 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:04:50.259059 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1b3fe0-fef8-4537-b724-f0d12bc767fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:04:50.259702 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1b3fe0-fef8-4537-b724-f0d12bc767fe service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:04:50.259979 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1b3fe0-fef8-4537-b724-f0d12bc767fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:04:50.260249 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1b3fe0-fef8-4537-b724-f0d12bc767fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:04:50.261860 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-6e1b3fe0-fef8-4537-b724-f0d12bc767fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:04:50.382367 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1b3fe0-fef8-4537-b724-f0d12bc767fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:04:50.383968 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1b3fe0-fef8-4537-b724-f0d12bc767fe service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:04:50.403555 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 676/1346] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:04:50 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:04:54.486391 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cbf564-e6c7-4037-a305-7be2400d630b None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:04:54.487009 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cbf564-e6c7-4037-a305-7be2400d630b None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:04:54.487009 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cbf564-e6c7-4037-a305-7be2400d630b None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:04:54.488323 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 671/1347] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 17:04:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:04:54.493862 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f9879e-c531-45f8-83b6-e44067d416d0 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:04:54.494320 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f9879e-c531-45f8-83b6-e44067d416d0 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:04:54.494445 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f9879e-c531-45f8-83b6-e44067d416d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:04:54.510386 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:04:54.566202 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-a5f9879e-c531-45f8-83b6-e44067d416d0 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:04:54.567511 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-a5f9879e-c531-45f8-83b6-e44067d416d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:04:54.672410 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 677/1348] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 17:04:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:04:54.958042 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cbf564-e6c7-4037-a305-7be2400d630b None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:04:54.959620 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-32cbf564-e6c7-4037-a305-7be2400d630b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:04:55.025743 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:04:55.073636 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cbf564-e6c7-4037-a305-7be2400d630b None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:04:55.146735 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c8d4e9-ac1f-4f76-a77d-440d98e40958 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:04:55.147367 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8d4e9-ac1f-4f76-a77d-440d98e40958 service neutron] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:04:55.147649 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8d4e9-ac1f-4f76-a77d-440d98e40958 service neutron] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:04:55.147921 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8d4e9-ac1f-4f76-a77d-440d98e40958 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:04:55.149569 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-23c8d4e9-ac1f-4f76-a77d-440d98e40958 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:04:55.254833 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:04:55.257998 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c8d4e9-ac1f-4f76-a77d-440d98e40958 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:04:55.259494 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c8d4e9-ac1f-4f76-a77d-440d98e40958 service neutron] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:04:55.282499 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 672/1349] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:04:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:05:00.246978 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd5e4a9-8ba4-49ee-9a5a-3b0a37d31193 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:05:00.248140 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd5e4a9-8ba4-49ee-9a5a-3b0a37d31193 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:05:00.248280 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd5e4a9-8ba4-49ee-9a5a-3b0a37d31193 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:05:00.249859 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 678/1350] 10.222.0.82 () {54 vars in 921 bytes} [Thu Aug 18 17:05:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:05:00.255339 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4504841f-69ae-4676-813f-7afc7813586e None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:05:00.255755 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4504841f-69ae-4676-813f-7afc7813586e None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:05:00.255948 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4504841f-69ae-4676-813f-7afc7813586e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:05:00.383337 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-4504841f-69ae-4676-813f-7afc7813586e None None] MFA Rules not processed for user `29653ba1a2a74232972ef7a9000518b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:05:00.384478 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-4504841f-69ae-4676-813f-7afc7813586e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:05:00.505715 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 673/1351] 10.222.0.82 () {58 vars in 1018 bytes} [Thu Aug 18 17:05:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:05:04.563575 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a7164f-6e2d-43e6-87a9-d8ccb37eb572 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:05:04.564448 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a7164f-6e2d-43e6-87a9-d8ccb37eb572 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:05:04.564719 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a7164f-6e2d-43e6-87a9-d8ccb37eb572 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:05:04.566261 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 679/1352] 10.222.0.82 () {54 vars in 921 bytes} [Thu Aug 18 17:05:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:05:04.571541 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c45af1-9d1a-4793-9305-389a964ee85b None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:05:04.571987 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c45af1-9d1a-4793-9305-389a964ee85b None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:05:04.572077 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c45af1-9d1a-4793-9305-389a964ee85b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:05:04.627725 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:05:04.650062 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-46c45af1-9d1a-4793-9305-389a964ee85b None None] MFA Rules not processed for user `29653ba1a2a74232972ef7a9000518b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:05:04.651300 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-46c45af1-9d1a-4793-9305-389a964ee85b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:05:04.670180 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=59 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:05:04.771679 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 674/1353] 10.222.0.82 () {58 vars in 1018 bytes} [Thu Aug 18 17:05:04 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:05:06.216880 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a8e234-b2de-494b-bbdf-6d16a47e03c5 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:05:06.217429 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a8e234-b2de-494b-bbdf-6d16a47e03c5 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:05:06.217429 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a8e234-b2de-494b-bbdf-6d16a47e03c5 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:05:06.218990 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 680/1354] 10.222.0.82 () {54 vars in 921 bytes} [Thu Aug 18 17:05:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 17:05:06.224111 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059f4a3b-0efe-492f-84cc-3f11b7f8158d None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:05:06.224584 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059f4a3b-0efe-492f-84cc-3f11b7f8158d None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:05:06.224702 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059f4a3b-0efe-492f-84cc-3f11b7f8158d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:05:06.302432 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-059f4a3b-0efe-492f-84cc-3f11b7f8158d None None] MFA Rules not processed for user `29653ba1a2a74232972ef7a9000518b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:05:06.303674 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-059f4a3b-0efe-492f-84cc-3f11b7f8158d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:05:06.347619 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:05:06.422349 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 675/1355] 10.222.0.82 () {58 vars in 1018 bytes} [Thu Aug 18 17:05:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:05:09.824138 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3100f0-5872-4b37-90bd-ac5b91a954b6 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:05:09.824660 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3100f0-5872-4b37-90bd-ac5b91a954b6 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:05:09.824660 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3100f0-5872-4b37-90bd-ac5b91a954b6 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:05:09.826070 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 681/1356] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 17:05:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:05:09.831273 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf93501-adb6-4414-8649-b3f40d42fa32 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:05:09.831701 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf93501-adb6-4414-8649-b3f40d42fa32 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:05:09.831806 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf93501-adb6-4414-8649-b3f40d42fa32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:05:09.907031 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-6bf93501-adb6-4414-8649-b3f40d42fa32 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:05:09.908210 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-6bf93501-adb6-4414-8649-b3f40d42fa32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:05:10.034297 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 676/1357] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 17:05:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:05:10.358773 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cd4a3b-4dd2-42dd-abfc-9717d903fa01 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:05:10.359234 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cd4a3b-4dd2-42dd-abfc-9717d903fa01 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:05:10.359660 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cd4a3b-4dd2-42dd-abfc-9717d903fa01 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:05:10.361238 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 682/1358] 10.222.0.82 () {54 vars in 962 bytes} [Thu Aug 18 17:05:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:05:10.370007 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3528c49-fc9b-442f-a15d-cdb9b3b80ff5 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:05:10.370445 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3528c49-fc9b-442f-a15d-cdb9b3b80ff5 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:05:10.377797 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3528c49-fc9b-442f-a15d-cdb9b3b80ff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:05:10.392741 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:05:10.459642 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-d3528c49-fc9b-442f-a15d-cdb9b3b80ff5 None None] MFA Rules not processed for user `55723c756cae4e7ab636054db48de02d`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:05:10.461012 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-d3528c49-fc9b-442f-a15d-cdb9b3b80ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:05:10.584797 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 677/1359] 10.222.0.82 () {58 vars in 1059 bytes} [Thu Aug 18 17:05:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:05:10.591304 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cd4a3b-4dd2-42dd-abfc-9717d903fa01 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:05:10.592972 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-c1cd4a3b-4dd2-42dd-abfc-9717d903fa01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:05:10.747124 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cd4a3b-4dd2-42dd-abfc-9717d903fa01 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:05:10.835253 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:05:10.878441 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bef7bc-b21f-4479-bb0a-eefda50de9c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55723c756cae4e7ab636054db48de02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:05:10.879083 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bef7bc-b21f-4479-bb0a-eefda50de9c3 service nova] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:05:10.879367 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bef7bc-b21f-4479-bb0a-eefda50de9c3 service nova] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:05:10.879641 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bef7bc-b21f-4479-bb0a-eefda50de9c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:05:10.881225 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-c0bef7bc-b21f-4479-bb0a-eefda50de9c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:05:10.973788 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bef7bc-b21f-4479-bb0a-eefda50de9c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:05:10.975448 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bef7bc-b21f-4479-bb0a-eefda50de9c3 service nova] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:05:10.994661 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 683/1360] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:05:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:05:13.129003 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce7369e-af57-47ec-b939-89855ea088c7 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:05:13.129540 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce7369e-af57-47ec-b939-89855ea088c7 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:05:13.129540 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce7369e-af57-47ec-b939-89855ea088c7 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:05:13.131130 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 678/1361] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 17:05:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:05:13.136425 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391eec75-2b53-4465-91b7-6cb7b702d9a3 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:05:13.136936 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391eec75-2b53-4465-91b7-6cb7b702d9a3 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:05:13.137052 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391eec75-2b53-4465-91b7-6cb7b702d9a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:05:13.206586 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-391eec75-2b53-4465-91b7-6cb7b702d9a3 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:05:13.207740 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-391eec75-2b53-4465-91b7-6cb7b702d9a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:05:13.247831 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:05:13.310544 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 684/1362] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 17:05:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:05:13.601302 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce7369e-af57-47ec-b939-89855ea088c7 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:05:13.602802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-6ce7369e-af57-47ec-b939-89855ea088c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:05:13.669700 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:05:13.755885 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce7369e-af57-47ec-b939-89855ea088c7 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:05:13.883107 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ddc449-ec9e-4e6d-9e7e-452775b03109 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55723c756cae4e7ab636054db48de02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:05:13.883720 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ddc449-ec9e-4e6d-9e7e-452775b03109 service nova] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:05:13.883998 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ddc449-ec9e-4e6d-9e7e-452775b03109 service nova] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:05:13.884264 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ddc449-ec9e-4e6d-9e7e-452775b03109 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:05:13.885835 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-32ddc449-ec9e-4e6d-9e7e-452775b03109 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:05:13.910613 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:05:13.996105 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ddc449-ec9e-4e6d-9e7e-452775b03109 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:05:14.000127 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ddc449-ec9e-4e6d-9e7e-452775b03109 service nova] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:05:14.022141 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 679/1363] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:05:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:05:16.401267 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe47e2a-6afd-4e06-b282-6700aac6ba79 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:05:16.401824 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe47e2a-6afd-4e06-b282-6700aac6ba79 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:05:16.401824 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe47e2a-6afd-4e06-b282-6700aac6ba79 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:05:16.403759 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 685/1364] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 17:05:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:05:16.409149 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d162748d-c516-4835-8e32-dbf28a25c4ce None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:05:16.409583 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d162748d-c516-4835-8e32-dbf28a25c4ce None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:05:16.409784 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d162748d-c516-4835-8e32-dbf28a25c4ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:05:16.479800 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-d162748d-c516-4835-8e32-dbf28a25c4ce None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:05:16.481043 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-d162748d-c516-4835-8e32-dbf28a25c4ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:05:16.577966 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:05:16.583324 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 680/1365] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 17:05:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:05:16.872486 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c1049-f0cb-4208-82be-6ee4bf8d9036 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:05:16.872940 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c1049-f0cb-4208-82be-6ee4bf8d9036 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:05:16.873473 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c1049-f0cb-4208-82be-6ee4bf8d9036 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:05:16.875116 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 686/1366] 10.222.0.82 () {54 vars in 962 bytes} [Thu Aug 18 17:05:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:05:16.880628 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08fcaaa-c78f-4f54-b183-8c36ec19d6d4 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:05:16.885141 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08fcaaa-c78f-4f54-b183-8c36ec19d6d4 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:05:16.885141 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08fcaaa-c78f-4f54-b183-8c36ec19d6d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:05:16.961113 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-b08fcaaa-c78f-4f54-b183-8c36ec19d6d4 None None] MFA Rules not processed for user `55723c756cae4e7ab636054db48de02d`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:05:16.962492 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-b08fcaaa-c78f-4f54-b183-8c36ec19d6d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:05:17.090572 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 681/1367] 10.222.0.82 () {58 vars in 1059 bytes} [Thu Aug 18 17:05:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:05:17.099099 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2c1049-f0cb-4208-82be-6ee4bf8d9036 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:05:17.100715 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-1c2c1049-f0cb-4208-82be-6ee4bf8d9036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:05:17.280173 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2c1049-f0cb-4208-82be-6ee4bf8d9036 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:05:17.310838 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:05:17.408191 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52baeabf-61e5-41c8-b074-506e29db1d11 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55723c756cae4e7ab636054db48de02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:05:17.408962 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52baeabf-61e5-41c8-b074-506e29db1d11 service nova] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:05:17.409137 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52baeabf-61e5-41c8-b074-506e29db1d11 service nova] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:05:17.409416 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52baeabf-61e5-41c8-b074-506e29db1d11 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:05:17.411077 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-52baeabf-61e5-41c8-b074-506e29db1d11 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:05:17.515043 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52baeabf-61e5-41c8-b074-506e29db1d11 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:05:17.516658 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52baeabf-61e5-41c8-b074-506e29db1d11 service nova] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:05:17.541424 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 687/1368] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:05:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 17:05:18.095219 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e7cbc-9d0f-462a-ac40-0c4a5a538e50 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:05:18.095751 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e7cbc-9d0f-462a-ac40-0c4a5a538e50 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:05:18.095751 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e7cbc-9d0f-462a-ac40-0c4a5a538e50 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:05:18.097237 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 682/1369] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 17:05:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:05:18.102166 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb948dd5-0144-47bf-95cb-0c27aa20aa7a None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:05:18.102455 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb948dd5-0144-47bf-95cb-0c27aa20aa7a None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:05:18.102738 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb948dd5-0144-47bf-95cb-0c27aa20aa7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:05:18.137309 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:05:18.174541 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-bb948dd5-0144-47bf-95cb-0c27aa20aa7a None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:05:18.176383 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-bb948dd5-0144-47bf-95cb-0c27aa20aa7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:05:18.283248 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 688/1370] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 17:05:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:05:18.884251 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e686cd-50d1-44da-8fc1-f5318829a54f None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:05:18.884251 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e686cd-50d1-44da-8fc1-f5318829a54f None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:05:18.884251 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e686cd-50d1-44da-8fc1-f5318829a54f None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:05:18.884251 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 683/1371] 10.222.0.82 () {54 vars in 964 bytes} [Thu Aug 18 17:05:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:05:18.890276 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d50e0b-dbe7-40a4-a862-c79e8ecca42f None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:05:18.890276 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d50e0b-dbe7-40a4-a862-c79e8ecca42f None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:05:18.890276 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d50e0b-dbe7-40a4-a862-c79e8ecca42f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:05:18.975572 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-e9d50e0b-dbe7-40a4-a862-c79e8ecca42f None None] MFA Rules not processed for user `85876a26a22e4c2c882883f7e978e4f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:05:18.976899 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-e9d50e0b-dbe7-40a4-a862-c79e8ecca42f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:05:19.026758 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:05:19.058260 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 689/1372] 10.222.0.82 () {58 vars in 1061 bytes} [Thu Aug 18 17:05:18 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:05:19.067070 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e686cd-50d1-44da-8fc1-f5318829a54f None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:05:19.069357 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-33e686cd-50d1-44da-8fc1-f5318829a54f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:05:19.124832 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:05:19.181647 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e686cd-50d1-44da-8fc1-f5318829a54f None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:05:19.270406 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a2bef8-c8a9-4421-be41-e8715d0f91b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:05:19.271061 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a2bef8-c8a9-4421-be41-e8715d0f91b1 service cinder] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:05:19.271339 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a2bef8-c8a9-4421-be41-e8715d0f91b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:05:19.271613 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a2bef8-c8a9-4421-be41-e8715d0f91b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:05:19.273242 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-67a2bef8-c8a9-4421-be41-e8715d0f91b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:05:19.384038 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:05:19.398732 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a2bef8-c8a9-4421-be41-e8715d0f91b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:05:19.401400 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a2bef8-c8a9-4421-be41-e8715d0f91b1 service cinder] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:05:19.424724 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 684/1373] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:05:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:05:19.657471 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85253a63-ae29-4ae5-8551-94aefda5deb4 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:05:19.657691 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85253a63-ae29-4ae5-8551-94aefda5deb4 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:05:19.657971 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85253a63-ae29-4ae5-8551-94aefda5deb4 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:05:19.663470 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 690/1374] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 17:05:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:05:19.664849 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f9287-0d0c-4d31-9e75-6bb8e5651c7a None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:05:19.665151 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f9287-0d0c-4d31-9e75-6bb8e5651c7a None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:05:19.665434 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f9287-0d0c-4d31-9e75-6bb8e5651c7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:05:19.733128 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-659f9287-0d0c-4d31-9e75-6bb8e5651c7a None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:05:19.734348 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-659f9287-0d0c-4d31-9e75-6bb8e5651c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:05:19.834547 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 685/1375] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 17:05:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 17:05:20.129273 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85253a63-ae29-4ae5-8551-94aefda5deb4 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:05:20.130868 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-85253a63-ae29-4ae5-8551-94aefda5deb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:05:20.276485 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85253a63-ae29-4ae5-8551-94aefda5deb4 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:05:20.331504 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:05:20.393576 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e68a5dd-ac35-47ff-8add-704b99cfa006 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55723c756cae4e7ab636054db48de02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:05:20.394434 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e68a5dd-ac35-47ff-8add-704b99cfa006 service nova] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:05:20.394813 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e68a5dd-ac35-47ff-8add-704b99cfa006 service nova] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:05:20.395112 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e68a5dd-ac35-47ff-8add-704b99cfa006 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:05:20.396716 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-3e68a5dd-ac35-47ff-8add-704b99cfa006 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:05:20.488689 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e68a5dd-ac35-47ff-8add-704b99cfa006 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:05:20.490268 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e68a5dd-ac35-47ff-8add-704b99cfa006 service nova] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:05:20.508433 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 691/1376] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:05:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:05:22.532129 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5a642-06d8-484b-9f62-efc5fac5ac18 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:05:22.532587 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5a642-06d8-484b-9f62-efc5fac5ac18 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:05:22.532687 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5a642-06d8-484b-9f62-efc5fac5ac18 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:05:22.534541 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 686/1377] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 17:05:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:05:22.539457 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c4cac7-1d52-401c-8409-5ebb284d3ad8 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:05:22.539921 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c4cac7-1d52-401c-8409-5ebb284d3ad8 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:05:22.540018 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c4cac7-1d52-401c-8409-5ebb284d3ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:05:22.610541 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-25c4cac7-1d52-401c-8409-5ebb284d3ad8 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:05:22.612430 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-25c4cac7-1d52-401c-8409-5ebb284d3ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:05:22.618733 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:05:22.747471 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 692/1378] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 17:05:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:05:23.052068 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc5a642-06d8-484b-9f62-efc5fac5ac18 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:05:23.055437 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-4bc5a642-06d8-484b-9f62-efc5fac5ac18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:05:23.139685 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:05:23.237304 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc5a642-06d8-484b-9f62-efc5fac5ac18 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:05:23.358693 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b0f80b-05a3-4c73-96e7-71248dae8f04 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55723c756cae4e7ab636054db48de02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:05:23.359403 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0f80b-05a3-4c73-96e7-71248dae8f04 service nova] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:05:23.359678 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0f80b-05a3-4c73-96e7-71248dae8f04 service nova] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:05:23.359945 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0f80b-05a3-4c73-96e7-71248dae8f04 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:05:23.361504 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-82b0f80b-05a3-4c73-96e7-71248dae8f04 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:05:23.380968 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:05:23.457792 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b0f80b-05a3-4c73-96e7-71248dae8f04 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:05:23.459414 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b0f80b-05a3-4c73-96e7-71248dae8f04 service nova] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:05:23.478430 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 687/1379] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:05:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:05:25.556416 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8178323-71d0-4021-9047-8bb99fa683ec None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:05:25.556962 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8178323-71d0-4021-9047-8bb99fa683ec None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:05:25.556962 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8178323-71d0-4021-9047-8bb99fa683ec None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:05:25.559100 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 693/1380] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 17:05:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:05:25.564366 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff407ab-8378-4b0d-b006-893133a381be None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:05:25.564782 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff407ab-8378-4b0d-b006-893133a381be None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:05:25.564959 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff407ab-8378-4b0d-b006-893133a381be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:05:25.634682 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-eff407ab-8378-4b0d-b006-893133a381be None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:05:25.635817 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-eff407ab-8378-4b0d-b006-893133a381be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:05:25.735494 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:05:25.741746 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 688/1381] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 17:05:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:05:26.033677 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8178323-71d0-4021-9047-8bb99fa683ec None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:05:26.035441 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-e8178323-71d0-4021-9047-8bb99fa683ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:05:26.188040 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:05:26.195336 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8178323-71d0-4021-9047-8bb99fa683ec None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:05:26.320808 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabc9675-b3e6-4593-8d75-a03bd49d54ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55723c756cae4e7ab636054db48de02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:05:26.321415 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabc9675-b3e6-4593-8d75-a03bd49d54ab service nova] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:05:26.321946 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabc9675-b3e6-4593-8d75-a03bd49d54ab service nova] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:05:26.322228 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabc9675-b3e6-4593-8d75-a03bd49d54ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:05:26.324443 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-eabc9675-b3e6-4593-8d75-a03bd49d54ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:05:26.419881 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabc9675-b3e6-4593-8d75-a03bd49d54ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:05:26.421500 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabc9675-b3e6-4593-8d75-a03bd49d54ab service nova] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:05:26.432710 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:05:26.442338 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 694/1382] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:05:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:05:28.562970 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d677ec-d6f5-45bd-8639-c58a16f906f9 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:05:28.563503 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d677ec-d6f5-45bd-8639-c58a16f906f9 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:05:28.563503 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d677ec-d6f5-45bd-8639-c58a16f906f9 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:05:28.564872 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 689/1383] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 17:05:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:05:28.570374 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71db938-581c-4f2b-b6aa-2878ead20c5f None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:05:28.570938 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71db938-581c-4f2b-b6aa-2878ead20c5f None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:05:28.570938 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71db938-581c-4f2b-b6aa-2878ead20c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:05:28.654740 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-c71db938-581c-4f2b-b6aa-2878ead20c5f None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:05:28.656262 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-c71db938-581c-4f2b-b6aa-2878ead20c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:05:28.765321 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 695/1384] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 17:05:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 17:05:29.068937 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d677ec-d6f5-45bd-8639-c58a16f906f9 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:05:29.071995 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-28d677ec-d6f5-45bd-8639-c58a16f906f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:05:29.245616 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d677ec-d6f5-45bd-8639-c58a16f906f9 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:05:29.335250 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:05:29.393212 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64e26af-b3e9-4770-ae5b-d9802aeb614f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55723c756cae4e7ab636054db48de02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:05:29.393826 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e26af-b3e9-4770-ae5b-d9802aeb614f service nova] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:05:29.394218 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e26af-b3e9-4770-ae5b-d9802aeb614f service nova] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:05:29.394489 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e26af-b3e9-4770-ae5b-d9802aeb614f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:05:29.396124 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-f64e26af-b3e9-4770-ae5b-d9802aeb614f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:05:29.490117 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64e26af-b3e9-4770-ae5b-d9802aeb614f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:05:29.491768 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64e26af-b3e9-4770-ae5b-d9802aeb614f service nova] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:05:29.510702 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 690/1385] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:05:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:05:31.434349 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c61bf9-843f-48d8-8cbc-2d01be0d9c79 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:05:31.435074 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c61bf9-843f-48d8-8cbc-2d01be0d9c79 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:05:31.435074 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c61bf9-843f-48d8-8cbc-2d01be0d9c79 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:05:31.436288 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 696/1386] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 17:05:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:05:31.441623 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421750d7-7fde-421b-a263-3ff04a4000dc None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:05:31.442149 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421750d7-7fde-421b-a263-3ff04a4000dc None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:05:31.442254 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421750d7-7fde-421b-a263-3ff04a4000dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:05:31.513952 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-421750d7-7fde-421b-a263-3ff04a4000dc None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:05:31.515617 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-421750d7-7fde-421b-a263-3ff04a4000dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:05:31.554261 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:05:31.618862 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 691/1387] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 17:05:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:05:31.910805 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c61bf9-843f-48d8-8cbc-2d01be0d9c79 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:05:31.912870 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-28c61bf9-843f-48d8-8cbc-2d01be0d9c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:05:32.007143 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:05:32.078861 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c61bf9-843f-48d8-8cbc-2d01be0d9c79 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:05:32.194665 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99276eaf-3b11-4246-bff3-ff1a6af74c31 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55723c756cae4e7ab636054db48de02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:05:32.195314 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99276eaf-3b11-4246-bff3-ff1a6af74c31 service nova] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:05:32.195598 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99276eaf-3b11-4246-bff3-ff1a6af74c31 service nova] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:05:32.195872 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99276eaf-3b11-4246-bff3-ff1a6af74c31 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:05:32.197748 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-99276eaf-3b11-4246-bff3-ff1a6af74c31 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:05:32.237710 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:05:32.289004 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99276eaf-3b11-4246-bff3-ff1a6af74c31 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:05:32.290591 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99276eaf-3b11-4246-bff3-ff1a6af74c31 service nova] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:05:32.310584 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 697/1388] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:05:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:05:34.344604 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c9b7f7-8df0-4f8c-aaf5-321e982f42fa None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:05:34.345146 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c9b7f7-8df0-4f8c-aaf5-321e982f42fa None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:05:34.345146 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c9b7f7-8df0-4f8c-aaf5-321e982f42fa None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:05:34.346512 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 692/1389] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 17:05:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:05:34.352191 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d14da34-c8a7-4c0d-95b3-33f655bdbb77 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:05:34.352685 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d14da34-c8a7-4c0d-95b3-33f655bdbb77 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:05:34.352775 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d14da34-c8a7-4c0d-95b3-33f655bdbb77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:05:34.426738 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-6d14da34-c8a7-4c0d-95b3-33f655bdbb77 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:05:34.428138 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-6d14da34-c8a7-4c0d-95b3-33f655bdbb77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:05:34.535015 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 698/1390] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 17:05:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 17:05:34.821889 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c9b7f7-8df0-4f8c-aaf5-321e982f42fa None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:05:34.824106 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-59c9b7f7-8df0-4f8c-aaf5-321e982f42fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:05:34.971666 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c9b7f7-8df0-4f8c-aaf5-321e982f42fa None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:05:34.988105 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:05:35.086397 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1d098e-60b7-49c2-9ce6-15d7a995c33d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55723c756cae4e7ab636054db48de02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:05:35.086943 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d098e-60b7-49c2-9ce6-15d7a995c33d service nova] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:05:35.087224 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d098e-60b7-49c2-9ce6-15d7a995c33d service nova] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:05:35.087488 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d098e-60b7-49c2-9ce6-15d7a995c33d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:05:35.089094 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-cd1d098e-60b7-49c2-9ce6-15d7a995c33d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:05:35.178392 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1d098e-60b7-49c2-9ce6-15d7a995c33d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:05:35.180026 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1d098e-60b7-49c2-9ce6-15d7a995c33d service nova] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:05:35.199825 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 693/1391] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:05:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:05:35.475682 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6931b3-14a1-4882-bd7b-5d0a8dc8d24d None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:05:35.476219 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6931b3-14a1-4882-bd7b-5d0a8dc8d24d None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:05:35.476219 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6931b3-14a1-4882-bd7b-5d0a8dc8d24d None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:05:35.477582 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 699/1392] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 17:05:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:05:35.483874 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c380d7-ad86-412f-850c-3cf8052d91fa None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:05:35.484396 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c380d7-ad86-412f-850c-3cf8052d91fa None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:05:35.484483 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c380d7-ad86-412f-850c-3cf8052d91fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:05:35.512511 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:05:35.558086 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-a6c380d7-ad86-412f-850c-3cf8052d91fa None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:05:35.559539 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-a6c380d7-ad86-412f-850c-3cf8052d91fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:05:35.656226 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 694/1393] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 17:05:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:05:35.959941 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b207ab84-9d9b-4fde-8181-7290816c919b None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:05:35.960227 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b207ab84-9d9b-4fde-8181-7290816c919b None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:05:35.960501 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b207ab84-9d9b-4fde-8181-7290816c919b None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:05:35.961824 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 700/1394] 10.222.0.82 () {54 vars in 967 bytes} [Thu Aug 18 17:05:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:05:35.967838 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2d527f-f163-46ba-8f5b-572691c0b24e None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:05:35.968130 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2d527f-f163-46ba-8f5b-572691c0b24e None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:05:35.968410 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2d527f-f163-46ba-8f5b-572691c0b24e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:05:36.043966 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-ae2d527f-f163-46ba-8f5b-572691c0b24e None None] MFA Rules not processed for user `2bee1ba6bb324de4915904f83349b3c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:05:36.045175 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-ae2d527f-f163-46ba-8f5b-572691c0b24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:05:36.086868 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:05:36.124405 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 695/1395] 10.222.0.82 () {58 vars in 1064 bytes} [Thu Aug 18 17:05:35 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:05:36.132436 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b207ab84-9d9b-4fde-8181-7290816c919b None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:05:36.134351 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-b207ab84-9d9b-4fde-8181-7290816c919b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:05:36.148987 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:05:36.244102 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b207ab84-9d9b-4fde-8181-7290816c919b None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:05:36.319899 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3373ecf8-7cb9-45a0-974e-b2a6872497b9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bee1ba6bb324de4915904f83349b3c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:05:36.320475 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3373ecf8-7cb9-45a0-974e-b2a6872497b9 service glance] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:05:36.320759 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3373ecf8-7cb9-45a0-974e-b2a6872497b9 service glance] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:05:36.321031 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3373ecf8-7cb9-45a0-974e-b2a6872497b9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:05:36.322926 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-3373ecf8-7cb9-45a0-974e-b2a6872497b9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:05:36.394278 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:05:36.436973 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3373ecf8-7cb9-45a0-974e-b2a6872497b9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:05:36.438530 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3373ecf8-7cb9-45a0-974e-b2a6872497b9 service glance] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:05:36.458827 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 701/1396] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:05:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:05:37.278078 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42796bd4-d95d-49b6-ac05-b742883f763c None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:05:37.278605 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42796bd4-d95d-49b6-ac05-b742883f763c None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:05:37.278605 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42796bd4-d95d-49b6-ac05-b742883f763c None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:05:37.280038 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 696/1397] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 17:05:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 17:05:37.286666 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb91e01-15f9-4a6d-9107-bb2a13455800 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:05:37.287317 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb91e01-15f9-4a6d-9107-bb2a13455800 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:05:37.288322 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb91e01-15f9-4a6d-9107-bb2a13455800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:05:37.305141 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e5430-f816-4f7f-804f-880ae55f7ca8 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:05:37.305417 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e5430-f816-4f7f-804f-880ae55f7ca8 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:05:37.305687 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e5430-f816-4f7f-804f-880ae55f7ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:05:37.359144 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-efb91e01-15f9-4a6d-9107-bb2a13455800 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:05:37.360421 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-efb91e01-15f9-4a6d-9107-bb2a13455800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:05:37.376827 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-853e5430-f816-4f7f-804f-880ae55f7ca8 None None] MFA Rules not processed for user `5de19a54ed104bcdb35ee4e5e793f1e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:05:37.378024 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-853e5430-f816-4f7f-804f-880ae55f7ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:05:37.461124 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 702/1398] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 17:05:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:05:37.467137 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 697/1399] 10.222.0.82 () {58 vars in 1009 bytes} [Thu Aug 18 17:05:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 17:05:37.509157 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a590ec-289f-4e18-a4cc-6c15bbc5494e None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:05:37.509518 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a590ec-289f-4e18-a4cc-6c15bbc5494e None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:05:37.509862 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a590ec-289f-4e18-a4cc-6c15bbc5494e None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:05:37.511972 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 703/1400] 10.222.0.82 () {54 vars in 943 bytes} [Thu Aug 18 17:05:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:05:37.518320 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be03bdc3-0139-4143-b289-ddbeedf62e4b None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:05:37.518674 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be03bdc3-0139-4143-b289-ddbeedf62e4b None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:05:37.519044 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be03bdc3-0139-4143-b289-ddbeedf62e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:05:37.587031 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:05:37.595569 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-be03bdc3-0139-4143-b289-ddbeedf62e4b None None] MFA Rules not processed for user `1381673f81584f4b8084823e3aa99e2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:05:37.599029 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-be03bdc3-0139-4143-b289-ddbeedf62e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:05:37.703751 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 698/1401] 10.222.0.82 () {58 vars in 1040 bytes} [Thu Aug 18 17:05:37 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 17:05:37.713314 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a590ec-289f-4e18-a4cc-6c15bbc5494e None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:05:37.715045 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-c9a590ec-289f-4e18-a4cc-6c15bbc5494e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:05:37.738039 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:05:37.751207 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be03bdc3-0139-4143-b289-ddbeedf62e4b None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:05:37.752397 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-be03bdc3-0139-4143-b289-ddbeedf62e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:05:37.855660 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be03bdc3-0139-4143-b289-ddbeedf62e4b None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:05:37.857449 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a590ec-289f-4e18-a4cc-6c15bbc5494e None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:05:37.879046 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:05:37.962336 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83b68a0-c966-4ec3-a19a-d22dd989afa3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1381673f81584f4b8084823e3aa99e2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:05:37.962969 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b68a0-c966-4ec3-a19a-d22dd989afa3 service swift] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:05:37.963247 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b68a0-c966-4ec3-a19a-d22dd989afa3 service swift] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:05:37.963526 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b68a0-c966-4ec3-a19a-d22dd989afa3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:05:37.965627 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-b83b68a0-c966-4ec3-a19a-d22dd989afa3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:05:37.972784 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:05:37.973850 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feab540-2c1c-4da4-8f94-593eaf46ffdd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55723c756cae4e7ab636054db48de02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:05:37.974463 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feab540-2c1c-4da4-8f94-593eaf46ffdd service nova] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:05:37.974737 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feab540-2c1c-4da4-8f94-593eaf46ffdd service nova] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:05:37.975047 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feab540-2c1c-4da4-8f94-593eaf46ffdd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:05:37.976684 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-5feab540-2c1c-4da4-8f94-593eaf46ffdd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:05:38.050628 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83b68a0-c966-4ec3-a19a-d22dd989afa3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:05:38.052302 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83b68a0-c966-4ec3-a19a-d22dd989afa3 service swift] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:05:38.061418 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 704/1402] 10.222.0.82 () {60 vars in 1384 bytes} [Thu Aug 18 17:05:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 17:05:38.074701 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5feab540-2c1c-4da4-8f94-593eaf46ffdd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:05:38.074701 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5feab540-2c1c-4da4-8f94-593eaf46ffdd service nova] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:05:38.090073 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 699/1403] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:05:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 17:05:40.344430 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d8b144-0f88-4c72-8701-9ff76a914c99 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:05:40.349393 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d8b144-0f88-4c72-8701-9ff76a914c99 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:05:40.349393 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d8b144-0f88-4c72-8701-9ff76a914c99 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:05:40.349393 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 705/1404] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 17:05:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:05:40.351504 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5e218b-4487-4957-aa49-2eb92764ccd7 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:05:40.351920 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5e218b-4487-4957-aa49-2eb92764ccd7 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:05:40.352072 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5e218b-4487-4957-aa49-2eb92764ccd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:05:40.386278 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:05:40.428496 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-cf5e218b-4487-4957-aa49-2eb92764ccd7 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:05:40.429722 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-cf5e218b-4487-4957-aa49-2eb92764ccd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:05:40.537791 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 700/1405] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 17:05:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:05:40.829897 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d8b144-0f88-4c72-8701-9ff76a914c99 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:05:40.831529 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-43d8b144-0f88-4c72-8701-9ff76a914c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:05:40.997328 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d8b144-0f88-4c72-8701-9ff76a914c99 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:05:41.000773 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:05:41.131580 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3319450-f903-42e5-8af3-9e2fec94c16a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55723c756cae4e7ab636054db48de02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:05:41.132195 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3319450-f903-42e5-8af3-9e2fec94c16a service nova] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:05:41.132748 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3319450-f903-42e5-8af3-9e2fec94c16a service nova] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:05:41.133026 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3319450-f903-42e5-8af3-9e2fec94c16a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:05:41.134776 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-d3319450-f903-42e5-8af3-9e2fec94c16a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:05:41.239290 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3319450-f903-42e5-8af3-9e2fec94c16a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:05:41.240910 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3319450-f903-42e5-8af3-9e2fec94c16a service nova] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:05:41.260709 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 706/1406] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:05:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:05:43.315704 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f064a37-741c-4f5d-83f1-14769d74f224 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:05:43.316275 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f064a37-741c-4f5d-83f1-14769d74f224 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:05:43.316275 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f064a37-741c-4f5d-83f1-14769d74f224 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:05:43.317597 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 701/1407] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 17:05:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:05:43.324091 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d07a29-662b-4e77-babe-c67902df98c3 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:05:43.325300 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d07a29-662b-4e77-babe-c67902df98c3 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:05:43.325759 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d07a29-662b-4e77-babe-c67902df98c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:05:43.336276 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:05:43.397801 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-51d07a29-662b-4e77-babe-c67902df98c3 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:05:43.399149 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-51d07a29-662b-4e77-babe-c67902df98c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:05:43.506948 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 707/1408] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 17:05:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:05:43.795104 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f064a37-741c-4f5d-83f1-14769d74f224 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:05:43.796760 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-1f064a37-741c-4f5d-83f1-14769d74f224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:05:43.907017 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:05:43.962457 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f064a37-741c-4f5d-83f1-14769d74f224 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:05:44.093112 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d39e46-bba5-413b-9eb9-472b91a683c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55723c756cae4e7ab636054db48de02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:05:44.093785 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d39e46-bba5-413b-9eb9-472b91a683c8 service nova] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:05:44.094069 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d39e46-bba5-413b-9eb9-472b91a683c8 service nova] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:05:44.094344 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d39e46-bba5-413b-9eb9-472b91a683c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:05:44.096027 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-e5d39e46-bba5-413b-9eb9-472b91a683c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:05:44.186826 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:05:44.222255 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d39e46-bba5-413b-9eb9-472b91a683c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:05:44.222255 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d39e46-bba5-413b-9eb9-472b91a683c8 service nova] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:05:44.247110 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 702/1409] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:05:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:05:53.501625 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=62 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:05:54.242876 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=16 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:06:01.596336 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6e0a9-22d4-4187-91f1-a0aaed36cd8c None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:06:01.596939 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6e0a9-22d4-4187-91f1-a0aaed36cd8c None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:06:01.596939 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6e0a9-22d4-4187-91f1-a0aaed36cd8c None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:06:01.598225 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 708/1410] 10.222.0.82 () {54 vars in 918 bytes} [Thu Aug 18 17:06:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:06:01.603340 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23848e-cccc-4ca6-8394-5c4205d39689 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:06:01.603791 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23848e-cccc-4ca6-8394-5c4205d39689 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:06:01.603894 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23848e-cccc-4ca6-8394-5c4205d39689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:06:01.677310 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-2e23848e-cccc-4ca6-8394-5c4205d39689 None None] MFA Rules not processed for user `29653ba1a2a74232972ef7a9000518b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:06:01.678518 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-2e23848e-cccc-4ca6-8394-5c4205d39689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:06:01.784969 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 703/1411] 10.222.0.82 () {58 vars in 1015 bytes} [Thu Aug 18 17:06:01 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:06:01.799800 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18015dc-7099-4a56-833d-a2892cada5b4 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:06:01.800087 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18015dc-7099-4a56-833d-a2892cada5b4 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:06:01.800365 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18015dc-7099-4a56-833d-a2892cada5b4 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:06:01.801699 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 709/1412] 10.222.0.82 () {54 vars in 918 bytes} [Thu Aug 18 17:06:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:06:01.806269 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bfeb3b-11ac-46e4-8343-dcf90f8e312d None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:06:01.806558 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bfeb3b-11ac-46e4-8343-dcf90f8e312d None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:06:01.806838 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bfeb3b-11ac-46e4-8343-dcf90f8e312d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:06:01.878862 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:06:01.882156 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-34bfeb3b-11ac-46e4-8343-dcf90f8e312d None None] MFA Rules not processed for user `55723c756cae4e7ab636054db48de02d`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:06:01.883459 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-34bfeb3b-11ac-46e4-8343-dcf90f8e312d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:06:02.000020 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 704/1413] 10.222.0.82 () {58 vars in 1015 bytes} [Thu Aug 18 17:06:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:06:04.379778 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729d8814-6e10-42bd-9fe2-43f3a2b4356f None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:06:04.380345 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729d8814-6e10-42bd-9fe2-43f3a2b4356f None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:06:04.380345 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729d8814-6e10-42bd-9fe2-43f3a2b4356f None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:06:04.381675 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 710/1414] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 17:06:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:06:04.387742 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6548ef-9caa-477d-909a-8b327e266eb5 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:06:04.388173 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6548ef-9caa-477d-909a-8b327e266eb5 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:06:04.388305 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6548ef-9caa-477d-909a-8b327e266eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:06:04.454173 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-3b6548ef-9caa-477d-909a-8b327e266eb5 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:06:04.455406 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-3b6548ef-9caa-477d-909a-8b327e266eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:06:04.509592 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:06:04.550764 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 705/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 17:06:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:06:04.840743 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729d8814-6e10-42bd-9fe2-43f3a2b4356f None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:06:04.842248 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-729d8814-6e10-42bd-9fe2-43f3a2b4356f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:06:04.987708 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729d8814-6e10-42bd-9fe2-43f3a2b4356f None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:06:05.060364 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:06:05.100665 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29574044-3485-4540-a3f2-70cf9152cfa5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55723c756cae4e7ab636054db48de02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:06:05.101270 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29574044-3485-4540-a3f2-70cf9152cfa5 service nova] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:06:05.101548 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29574044-3485-4540-a3f2-70cf9152cfa5 service nova] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:06:05.101817 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29574044-3485-4540-a3f2-70cf9152cfa5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:06:05.103951 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-29574044-3485-4540-a3f2-70cf9152cfa5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:06:05.193130 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29574044-3485-4540-a3f2-70cf9152cfa5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:06:05.194738 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29574044-3485-4540-a3f2-70cf9152cfa5 service nova] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:06:05.212842 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 711/1416] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:06:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:06:06.110624 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd0d29-bd8f-4a80-a538-c24c74601ff2 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:06:06.111204 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd0d29-bd8f-4a80-a538-c24c74601ff2 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:06:06.111204 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd0d29-bd8f-4a80-a538-c24c74601ff2 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:06:06.112525 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 706/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 18 17:06:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:06:06.117370 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e612707f-9b7f-42df-9f3b-affdd6632885 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:06:06.117796 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e612707f-9b7f-42df-9f3b-affdd6632885 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:06:06.117924 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e612707f-9b7f-42df-9f3b-affdd6632885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:06:06.194599 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-e612707f-9b7f-42df-9f3b-affdd6632885 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:06:06.196107 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-e612707f-9b7f-42df-9f3b-affdd6632885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:06:06.223341 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:06:06.293295 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 712/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 18 17:06:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:06:06.445226 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dd0d29-bd8f-4a80-a538-c24c74601ff2 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:06:06.447553 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-25dd0d29-bd8f-4a80-a538-c24c74601ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:06:06.590034 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dd0d29-bd8f-4a80-a538-c24c74601ff2 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:06:06.630033 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:06:06.701084 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268397ad-0bbd-4708-a635-f57e4b0b2b05 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55723c756cae4e7ab636054db48de02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:06:06.701612 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268397ad-0bbd-4708-a635-f57e4b0b2b05 service nova] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:06:06.701888 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268397ad-0bbd-4708-a635-f57e4b0b2b05 service nova] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:06:06.702162 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268397ad-0bbd-4708-a635-f57e4b0b2b05 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:06:06.703755 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-268397ad-0bbd-4708-a635-f57e4b0b2b05 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:06:06.792168 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268397ad-0bbd-4708-a635-f57e4b0b2b05 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:06:06.793775 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268397ad-0bbd-4708-a635-f57e4b0b2b05 service nova] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:06:06.812096 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 707/1419] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:06:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:06:07.690468 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c544fa-5340-474d-bcbb-b256b5b5db5d None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:06:07.691001 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c544fa-5340-474d-bcbb-b256b5b5db5d None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:06:07.691001 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c544fa-5340-474d-bcbb-b256b5b5db5d None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:06:07.692292 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 713/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 18 17:06:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:06:07.698033 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1766537-3163-43d2-b5f6-b5a00673a904 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:06:07.698438 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1766537-3163-43d2-b5f6-b5a00673a904 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:06:07.698592 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1766537-3163-43d2-b5f6-b5a00673a904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:06:07.751791 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:06:07.774138 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-b1766537-3163-43d2-b5f6-b5a00673a904 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:06:07.776134 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-b1766537-3163-43d2-b5f6-b5a00673a904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:06:07.871493 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 708/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 18 17:06:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:06:08.024971 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c544fa-5340-474d-bcbb-b256b5b5db5d None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:06:08.026457 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-a9c544fa-5340-474d-bcbb-b256b5b5db5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:06:08.170260 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c544fa-5340-474d-bcbb-b256b5b5db5d None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:06:08.183036 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:06:08.282583 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f00dda-7659-4291-ab10-2a589354a2e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55723c756cae4e7ab636054db48de02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:06:08.283137 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f00dda-7659-4291-ab10-2a589354a2e7 service nova] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:06:08.283421 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f00dda-7659-4291-ab10-2a589354a2e7 service nova] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:06:08.283694 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f00dda-7659-4291-ab10-2a589354a2e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:06:08.285307 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-c9f00dda-7659-4291-ab10-2a589354a2e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:06:08.374644 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f00dda-7659-4291-ab10-2a589354a2e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:06:08.376281 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f00dda-7659-4291-ab10-2a589354a2e7 service nova] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:06:08.393913 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 714/1422] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:06:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:06:09.260276 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41f390-a4af-47dc-aba8-aa015f01a05d None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:06:09.260864 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41f390-a4af-47dc-aba8-aa015f01a05d None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:06:09.260864 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41f390-a4af-47dc-aba8-aa015f01a05d None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:06:09.262229 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 709/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 18 17:06:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:06:09.267075 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db10ae-43a3-45bf-a979-17ce3f7cf853 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:06:09.267517 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db10ae-43a3-45bf-a979-17ce3f7cf853 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:06:09.267628 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db10ae-43a3-45bf-a979-17ce3f7cf853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:06:09.287359 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:06:09.343715 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-b3db10ae-43a3-45bf-a979-17ce3f7cf853 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:06:09.344924 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-b3db10ae-43a3-45bf-a979-17ce3f7cf853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:06:09.439941 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 715/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 18 17:06:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:06:09.588422 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e41f390-a4af-47dc-aba8-aa015f01a05d None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:06:09.589953 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-0e41f390-a4af-47dc-aba8-aa015f01a05d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:06:09.700188 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:06:09.736054 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e41f390-a4af-47dc-aba8-aa015f01a05d None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:06:09.849671 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7f2530-01f6-4fbd-b365-e2d91fa45395 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55723c756cae4e7ab636054db48de02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:06:09.850203 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f2530-01f6-4fbd-b365-e2d91fa45395 service nova] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:06:09.850483 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f2530-01f6-4fbd-b365-e2d91fa45395 service nova] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:06:09.850754 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f2530-01f6-4fbd-b365-e2d91fa45395 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:06:09.852861 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-db7f2530-01f6-4fbd-b365-e2d91fa45395 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:06:09.928708 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:06:09.942692 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7f2530-01f6-4fbd-b365-e2d91fa45395 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:06:09.944338 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7f2530-01f6-4fbd-b365-e2d91fa45395 service nova] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:06:09.963371 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 710/1425] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:06:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 17:06:10.828500 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d50d77b-cf31-442d-9def-3bb7015b3ba9 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:06:10.829202 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d50d77b-cf31-442d-9def-3bb7015b3ba9 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:06:10.829202 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d50d77b-cf31-442d-9def-3bb7015b3ba9 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:06:10.830388 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 716/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 18 17:06:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:06:10.835308 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0722ca9-7150-46d6-b113-97d13e201ce1 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:06:10.835845 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0722ca9-7150-46d6-b113-97d13e201ce1 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:06:10.835845 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0722ca9-7150-46d6-b113-97d13e201ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:06:10.912789 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-b0722ca9-7150-46d6-b113-97d13e201ce1 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:06:10.913990 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-b0722ca9-7150-46d6-b113-97d13e201ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:06:11.011229 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 711/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 18 17:06:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:06:11.163261 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d50d77b-cf31-442d-9def-3bb7015b3ba9 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:06:11.164834 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-5d50d77b-cf31-442d-9def-3bb7015b3ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:06:11.248565 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:06:11.310927 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d50d77b-cf31-442d-9def-3bb7015b3ba9 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:06:11.421756 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14cc320-b753-4a7f-9995-c8ee6af4df53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55723c756cae4e7ab636054db48de02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:06:11.422370 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14cc320-b753-4a7f-9995-c8ee6af4df53 service nova] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:06:11.422651 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14cc320-b753-4a7f-9995-c8ee6af4df53 service nova] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:06:11.422949 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14cc320-b753-4a7f-9995-c8ee6af4df53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:06:11.424555 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-e14cc320-b753-4a7f-9995-c8ee6af4df53 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:06:11.472197 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:06:11.512466 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14cc320-b753-4a7f-9995-c8ee6af4df53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:06:11.514065 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14cc320-b753-4a7f-9995-c8ee6af4df53 service nova] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:06:11.531776 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 717/1428] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:06:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:06:12.394841 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f160e5-a457-4b07-8fe7-383cb3a5f9a5 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:06:12.395437 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f160e5-a457-4b07-8fe7-383cb3a5f9a5 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:06:12.395437 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f160e5-a457-4b07-8fe7-383cb3a5f9a5 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:06:12.396758 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 712/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 18 17:06:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:06:12.402351 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb825861-d462-43a3-adbc-c73e53308263 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:06:12.402745 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb825861-d462-43a3-adbc-c73e53308263 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:06:12.402916 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb825861-d462-43a3-adbc-c73e53308263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:06:12.477996 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-cb825861-d462-43a3-adbc-c73e53308263 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:06:12.479450 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-cb825861-d462-43a3-adbc-c73e53308263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:06:12.574463 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 718/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 18 17:06:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:06:12.725176 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f160e5-a457-4b07-8fe7-383cb3a5f9a5 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:06:12.726651 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-b7f160e5-a457-4b07-8fe7-383cb3a5f9a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:06:12.775986 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:06:12.872602 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f160e5-a457-4b07-8fe7-383cb3a5f9a5 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:06:12.984086 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5a5f82-76b8-41c2-bbc7-98c1704e5865 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55723c756cae4e7ab636054db48de02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:06:12.984618 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a5f82-76b8-41c2-bbc7-98c1704e5865 service nova] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:06:12.984904 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a5f82-76b8-41c2-bbc7-98c1704e5865 service nova] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:06:12.985176 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a5f82-76b8-41c2-bbc7-98c1704e5865 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:06:12.986781 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-6c5a5f82-76b8-41c2-bbc7-98c1704e5865 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:06:12.993832 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:06:13.075915 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5a5f82-76b8-41c2-bbc7-98c1704e5865 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:06:13.077526 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5a5f82-76b8-41c2-bbc7-98c1704e5865 service nova] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:06:13.095569 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 713/1431] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:06:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:06:13.950859 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32890465-40c3-48b2-8b09-41edfa36bcd4 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:06:13.951358 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32890465-40c3-48b2-8b09-41edfa36bcd4 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:06:13.951463 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32890465-40c3-48b2-8b09-41edfa36bcd4 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:06:13.952787 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 719/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 18 17:06:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:06:13.957672 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8478372c-e129-4289-9f03-eee26dcd7711 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:06:13.958076 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8478372c-e129-4289-9f03-eee26dcd7711 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:06:13.958235 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8478372c-e129-4289-9f03-eee26dcd7711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:06:14.032881 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-8478372c-e129-4289-9f03-eee26dcd7711 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:06:14.033982 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-8478372c-e129-4289-9f03-eee26dcd7711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:06:14.100406 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:06:14.128515 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 714/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 18 17:06:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:06:14.283371 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32890465-40c3-48b2-8b09-41edfa36bcd4 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:06:14.284825 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-32890465-40c3-48b2-8b09-41edfa36bcd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:06:14.303060 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:06:14.431555 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32890465-40c3-48b2-8b09-41edfa36bcd4 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:06:14.634573 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:06:14.660700 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c145afa2-dfb9-4805-a2f2-ddec11a690e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55723c756cae4e7ab636054db48de02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:06:14.661303 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c145afa2-dfb9-4805-a2f2-ddec11a690e4 service nova] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:06:14.661578 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c145afa2-dfb9-4805-a2f2-ddec11a690e4 service nova] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:06:14.661842 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c145afa2-dfb9-4805-a2f2-ddec11a690e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:06:14.663449 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-c145afa2-dfb9-4805-a2f2-ddec11a690e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:06:14.752455 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c145afa2-dfb9-4805-a2f2-ddec11a690e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:06:14.754026 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c145afa2-dfb9-4805-a2f2-ddec11a690e4 service nova] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:06:14.771798 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 720/1434] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:06:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:06:15.631826 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0e32db-1ebe-4b8d-8d58-5a02703c929b None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:06:15.632392 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0e32db-1ebe-4b8d-8d58-5a02703c929b None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:06:15.632392 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0e32db-1ebe-4b8d-8d58-5a02703c929b None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:06:15.634244 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 715/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 18 17:06:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:06:15.639100 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec59e91b-7d09-4701-916f-a0533dfa8fce None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:06:15.639508 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec59e91b-7d09-4701-916f-a0533dfa8fce None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:06:15.639653 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec59e91b-7d09-4701-916f-a0533dfa8fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:06:15.713983 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-ec59e91b-7d09-4701-916f-a0533dfa8fce None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:06:15.715203 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-ec59e91b-7d09-4701-916f-a0533dfa8fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:06:15.755544 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:06:15.811489 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 721/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 18 17:06:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:06:15.965947 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0e32db-1ebe-4b8d-8d58-5a02703c929b None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:06:15.967496 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-ba0e32db-1ebe-4b8d-8d58-5a02703c929b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:06:16.110809 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0e32db-1ebe-4b8d-8d58-5a02703c929b None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:06:16.166559 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:06:16.226877 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f86b00-908a-429f-ada6-bdab91041081 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55723c756cae4e7ab636054db48de02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:06:16.227558 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f86b00-908a-429f-ada6-bdab91041081 service nova] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:06:16.227822 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f86b00-908a-429f-ada6-bdab91041081 service nova] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:06:16.228094 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f86b00-908a-429f-ada6-bdab91041081 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:06:16.229758 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-43f86b00-908a-429f-ada6-bdab91041081 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:06:16.320333 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f86b00-908a-429f-ada6-bdab91041081 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:06:16.322645 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f86b00-908a-429f-ada6-bdab91041081 service nova] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:06:16.340295 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 716/1437] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:06:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:06:17.218084 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7deb782-6f45-4320-8f7d-03a36f35f6c6 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:06:17.218660 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7deb782-6f45-4320-8f7d-03a36f35f6c6 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:06:17.218660 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7deb782-6f45-4320-8f7d-03a36f35f6c6 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:06:17.220012 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 722/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 18 17:06:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:06:17.224991 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd4d4a-66ad-436c-b056-0bfe8854ec77 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:06:17.225383 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd4d4a-66ad-436c-b056-0bfe8854ec77 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:06:17.225547 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd4d4a-66ad-436c-b056-0bfe8854ec77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:06:17.300490 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-10fd4d4a-66ad-436c-b056-0bfe8854ec77 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:06:17.301681 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-10fd4d4a-66ad-436c-b056-0bfe8854ec77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:06:17.303683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:06:17.400676 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 717/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 18 17:06:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:06:17.551479 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7deb782-6f45-4320-8f7d-03a36f35f6c6 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:06:17.553035 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-b7deb782-6f45-4320-8f7d-03a36f35f6c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:06:17.703342 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7deb782-6f45-4320-8f7d-03a36f35f6c6 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:06:17.728482 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:06:17.813264 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fa3675-bf7e-4751-bf01-403fb78e1921 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55723c756cae4e7ab636054db48de02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:06:17.813801 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fa3675-bf7e-4751-bf01-403fb78e1921 service nova] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:06:17.814082 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fa3675-bf7e-4751-bf01-403fb78e1921 service nova] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:06:17.814353 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fa3675-bf7e-4751-bf01-403fb78e1921 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:06:17.815991 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-99fa3675-bf7e-4751-bf01-403fb78e1921 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:06:17.903435 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fa3675-bf7e-4751-bf01-403fb78e1921 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:06:17.905040 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fa3675-bf7e-4751-bf01-403fb78e1921 service nova] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:06:17.923511 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 723/1440] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:06:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:06:19.740365 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a9d74b-8956-4f40-95c8-85091d24c6ed None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:06:19.740901 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a9d74b-8956-4f40-95c8-85091d24c6ed None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:06:19.740901 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a9d74b-8956-4f40-95c8-85091d24c6ed None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:06:19.742255 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 718/1441] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 17:06:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:06:19.748388 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b270eeb3-4c07-4def-a657-ab2a4af2b47b None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:06:19.748794 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b270eeb3-4c07-4def-a657-ab2a4af2b47b None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:06:19.748948 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b270eeb3-4c07-4def-a657-ab2a4af2b47b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:06:19.789781 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:06:19.814688 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-b270eeb3-4c07-4def-a657-ab2a4af2b47b None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:06:19.815926 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-b270eeb3-4c07-4def-a657-ab2a4af2b47b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:06:19.910747 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 724/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 17:06:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:06:20.192387 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a9d74b-8956-4f40-95c8-85091d24c6ed None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:06:20.193973 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-53a9d74b-8956-4f40-95c8-85091d24c6ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:06:20.298414 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a9d74b-8956-4f40-95c8-85091d24c6ed None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:06:20.330075 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:06:20.371921 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2c37c1-0ee4-41df-84a7-2abd8b72a409 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:06:20.372528 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2c37c1-0ee4-41df-84a7-2abd8b72a409 service neutron] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:06:20.372809 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2c37c1-0ee4-41df-84a7-2abd8b72a409 service neutron] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:06:20.373089 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2c37c1-0ee4-41df-84a7-2abd8b72a409 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:06:20.374718 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-cb2c37c1-0ee4-41df-84a7-2abd8b72a409 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:06:20.486975 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2c37c1-0ee4-41df-84a7-2abd8b72a409 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:06:20.488363 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2c37c1-0ee4-41df-84a7-2abd8b72a409 service neutron] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:06:20.507261 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 719/1443] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:06:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:06:22.564135 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc253dd-d1fe-46a8-a599-ed53aace42ae None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:06:22.564700 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc253dd-d1fe-46a8-a599-ed53aace42ae None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:06:22.564700 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc253dd-d1fe-46a8-a599-ed53aace42ae None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:06:22.566028 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 725/1444] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 17:06:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:06:22.572089 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c44e1a-5e02-407b-b784-15623dcb96e2 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:06:22.572515 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c44e1a-5e02-407b-b784-15623dcb96e2 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:06:22.572648 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c44e1a-5e02-407b-b784-15623dcb96e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:06:22.637606 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-e1c44e1a-5e02-407b-b784-15623dcb96e2 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:06:22.638852 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-e1c44e1a-5e02-407b-b784-15623dcb96e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:06:22.652319 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:06:22.735359 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 720/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 17:06:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:06:23.016560 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc253dd-d1fe-46a8-a599-ed53aace42ae None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:06:23.018091 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-1cc253dd-d1fe-46a8-a599-ed53aace42ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:06:23.119408 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc253dd-d1fe-46a8-a599-ed53aace42ae None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:06:23.131060 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:06:23.191001 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ca7159-0df7-40c2-b698-b5eeec142839 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:06:23.191604 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca7159-0df7-40c2-b698-b5eeec142839 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:06:23.191881 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca7159-0df7-40c2-b698-b5eeec142839 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:06:23.192154 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca7159-0df7-40c2-b698-b5eeec142839 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:06:23.193727 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-b1ca7159-0df7-40c2-b698-b5eeec142839 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:06:23.307207 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ca7159-0df7-40c2-b698-b5eeec142839 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:06:23.308640 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ca7159-0df7-40c2-b698-b5eeec142839 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:06:23.326229 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 726/1446] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:06:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:06:25.599375 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948ded81-ca1c-4c0c-8698-94adac402384 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:06:25.599939 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948ded81-ca1c-4c0c-8698-94adac402384 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:06:25.599939 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948ded81-ca1c-4c0c-8698-94adac402384 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:06:25.601271 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 721/1447] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 17:06:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:06:25.607152 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e30ac-4f22-4934-8446-f0c98de46013 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:06:25.607538 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e30ac-4f22-4934-8446-f0c98de46013 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:06:25.607699 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e30ac-4f22-4934-8446-f0c98de46013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:06:25.655141 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:06:25.672862 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-853e30ac-4f22-4934-8446-f0c98de46013 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:06:25.684812 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-853e30ac-4f22-4934-8446-f0c98de46013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:06:25.769269 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 727/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 17:06:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:06:26.052848 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948ded81-ca1c-4c0c-8698-94adac402384 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:06:26.054383 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-948ded81-ca1c-4c0c-8698-94adac402384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:06:26.157927 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948ded81-ca1c-4c0c-8698-94adac402384 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:06:26.198886 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:06:26.232153 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e36ab7-39fc-4578-914e-f8adee5df6a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:06:26.232760 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e36ab7-39fc-4578-914e-f8adee5df6a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:06:26.233037 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e36ab7-39fc-4578-914e-f8adee5df6a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:06:26.233311 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e36ab7-39fc-4578-914e-f8adee5df6a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:06:26.234889 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-90e36ab7-39fc-4578-914e-f8adee5df6a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:06:26.343657 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e36ab7-39fc-4578-914e-f8adee5df6a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:06:26.345102 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e36ab7-39fc-4578-914e-f8adee5df6a0 service neutron] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:06:26.364285 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 722/1449] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:06:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:06:28.394009 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c2f76-e0d8-4963-a3cd-b6c71c0804f7 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:06:28.394554 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c2f76-e0d8-4963-a3cd-b6c71c0804f7 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:06:28.394862 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c2f76-e0d8-4963-a3cd-b6c71c0804f7 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:06:28.396408 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 728/1450] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 17:06:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:06:28.401879 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d91b94-9596-41da-bbcd-2accc3a32092 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:06:28.402398 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d91b94-9596-41da-bbcd-2accc3a32092 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:06:28.402719 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d91b94-9596-41da-bbcd-2accc3a32092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:06:28.474184 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-10d91b94-9596-41da-bbcd-2accc3a32092 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:06:28.475429 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-10d91b94-9596-41da-bbcd-2accc3a32092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:06:28.497994 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:06:28.583174 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 723/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 17:06:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:06:28.891731 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03c2f76-e0d8-4963-a3cd-b6c71c0804f7 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:06:28.893659 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-a03c2f76-e0d8-4963-a3cd-b6c71c0804f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:06:29.013877 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03c2f76-e0d8-4963-a3cd-b6c71c0804f7 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:06:29.038417 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:06:29.092830 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a38034f-4ba5-4718-816a-7bce979d9288 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:06:29.093496 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a38034f-4ba5-4718-816a-7bce979d9288 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:06:29.093777 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a38034f-4ba5-4718-816a-7bce979d9288 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:06:29.094044 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a38034f-4ba5-4718-816a-7bce979d9288 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:06:29.095655 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-1a38034f-4ba5-4718-816a-7bce979d9288 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:06:29.204659 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a38034f-4ba5-4718-816a-7bce979d9288 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:06:29.206120 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a38034f-4ba5-4718-816a-7bce979d9288 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:06:29.224720 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 729/1452] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:06:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 17:06:31.457949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685ebcd1-5cdb-4833-a738-773e706d157c None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:06:31.458527 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685ebcd1-5cdb-4833-a738-773e706d157c None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:06:31.458527 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685ebcd1-5cdb-4833-a738-773e706d157c None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:06:31.459876 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 724/1453] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 17:06:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:06:31.465377 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6013c9cf-0a41-49ef-8bad-d5310f09ada8 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:06:31.465802 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6013c9cf-0a41-49ef-8bad-d5310f09ada8 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:06:31.465935 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6013c9cf-0a41-49ef-8bad-d5310f09ada8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:06:31.528574 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:06:31.531281 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-6013c9cf-0a41-49ef-8bad-d5310f09ada8 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:06:31.532481 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-6013c9cf-0a41-49ef-8bad-d5310f09ada8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:06:31.626557 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 730/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 17:06:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:06:31.911056 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685ebcd1-5cdb-4833-a738-773e706d157c None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:06:31.912620 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-685ebcd1-5cdb-4833-a738-773e706d157c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:06:32.017842 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685ebcd1-5cdb-4833-a738-773e706d157c None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:06:32.068214 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:06:32.093228 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeaa9e1-93d6-4679-ab29-229522e0858a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:06:32.093741 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeaa9e1-93d6-4679-ab29-229522e0858a service neutron] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:06:32.094028 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeaa9e1-93d6-4679-ab29-229522e0858a service neutron] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:06:32.094283 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeaa9e1-93d6-4679-ab29-229522e0858a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:06:32.095902 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-abeaa9e1-93d6-4679-ab29-229522e0858a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:06:32.202643 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeaa9e1-93d6-4679-ab29-229522e0858a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:06:32.204116 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeaa9e1-93d6-4679-ab29-229522e0858a service neutron] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:06:32.222376 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 725/1455] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:06:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:06:34.225602 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac496760-70c2-4f17-942c-2b700ac7c13e None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:06:34.226277 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac496760-70c2-4f17-942c-2b700ac7c13e None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:06:34.226371 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac496760-70c2-4f17-942c-2b700ac7c13e None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:06:34.227878 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 731/1456] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 17:06:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:06:34.233458 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb4454b-2bc9-4124-9e8e-aac242c05812 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:06:34.233844 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb4454b-2bc9-4124-9e8e-aac242c05812 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:06:34.234015 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb4454b-2bc9-4124-9e8e-aac242c05812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:06:34.301705 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-0cb4454b-2bc9-4124-9e8e-aac242c05812 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:06:34.302968 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-0cb4454b-2bc9-4124-9e8e-aac242c05812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:06:34.332671 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:06:34.404283 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 726/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 17:06:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 17:06:34.688957 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac496760-70c2-4f17-942c-2b700ac7c13e None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:06:34.690511 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-ac496760-70c2-4f17-942c-2b700ac7c13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:06:34.791634 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac496760-70c2-4f17-942c-2b700ac7c13e None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:06:34.819700 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:06:34.863475 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888bf07f-5fc5-4e3a-9532-7931c79b9d4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:06:34.864124 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888bf07f-5fc5-4e3a-9532-7931c79b9d4c service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:06:34.864607 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888bf07f-5fc5-4e3a-9532-7931c79b9d4c service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:06:34.865095 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888bf07f-5fc5-4e3a-9532-7931c79b9d4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:06:34.867963 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-888bf07f-5fc5-4e3a-9532-7931c79b9d4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:06:34.973728 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888bf07f-5fc5-4e3a-9532-7931c79b9d4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:06:34.975096 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888bf07f-5fc5-4e3a-9532-7931c79b9d4c service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:06:34.995089 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 732/1458] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:06:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:06:36.970824 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11733a7-8bcc-471f-b3cd-8536dc5c2038 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:06:36.971403 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11733a7-8bcc-471f-b3cd-8536dc5c2038 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:06:36.971403 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11733a7-8bcc-471f-b3cd-8536dc5c2038 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:06:36.972736 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 727/1459] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 17:06:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:06:36.978651 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67444147-095b-4d8d-8fb2-ae7e1abad52f None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:06:36.979080 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67444147-095b-4d8d-8fb2-ae7e1abad52f None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:06:36.979268 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67444147-095b-4d8d-8fb2-ae7e1abad52f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:06:37.046467 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-67444147-095b-4d8d-8fb2-ae7e1abad52f None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:06:37.047711 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-67444147-095b-4d8d-8fb2-ae7e1abad52f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:06:37.056285 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:06:37.144694 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 733/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 17:06:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:06:37.427560 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11733a7-8bcc-471f-b3cd-8536dc5c2038 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:06:37.429636 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-d11733a7-8bcc-471f-b3cd-8536dc5c2038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:06:37.531958 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11733a7-8bcc-471f-b3cd-8536dc5c2038 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:06:37.587432 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:06:37.605637 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e030c06-de25-4afb-999b-c1403c8331cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:06:37.612035 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e030c06-de25-4afb-999b-c1403c8331cc service neutron] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:06:37.612035 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e030c06-de25-4afb-999b-c1403c8331cc service neutron] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:06:37.612035 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e030c06-de25-4afb-999b-c1403c8331cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:06:37.612035 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-3e030c06-de25-4afb-999b-c1403c8331cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:06:37.714662 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e030c06-de25-4afb-999b-c1403c8331cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:06:37.716101 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e030c06-de25-4afb-999b-c1403c8331cc service neutron] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:06:37.734237 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 728/1461] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:06:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:06:39.673468 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f9f22-d6ee-4731-80a9-bfc639aeeb50 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:06:39.674034 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f9f22-d6ee-4731-80a9-bfc639aeeb50 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:06:39.674034 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f9f22-d6ee-4731-80a9-bfc639aeeb50 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:06:39.675369 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 734/1462] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 17:06:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:06:39.681095 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb221e5f-fa9f-4a0e-95ce-9a68b2ffc855 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:06:39.681532 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb221e5f-fa9f-4a0e-95ce-9a68b2ffc855 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:06:39.681630 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb221e5f-fa9f-4a0e-95ce-9a68b2ffc855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:06:39.747254 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-bb221e5f-fa9f-4a0e-95ce-9a68b2ffc855 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:06:39.748463 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-bb221e5f-fa9f-4a0e-95ce-9a68b2ffc855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:06:39.787645 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:06:39.846120 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 729/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 17:06:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 17:06:40.127767 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35f9f22-d6ee-4731-80a9-bfc639aeeb50 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:06:40.129308 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-e35f9f22-d6ee-4731-80a9-bfc639aeeb50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:06:40.232449 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35f9f22-d6ee-4731-80a9-bfc639aeeb50 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:06:40.269245 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:06:40.305115 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36ede28-107f-4d87-b2d9-22d6311e9ad5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:06:40.305723 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ede28-107f-4d87-b2d9-22d6311e9ad5 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:06:40.306023 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ede28-107f-4d87-b2d9-22d6311e9ad5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:06:40.306277 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ede28-107f-4d87-b2d9-22d6311e9ad5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:06:40.307914 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-f36ede28-107f-4d87-b2d9-22d6311e9ad5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:06:40.417167 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36ede28-107f-4d87-b2d9-22d6311e9ad5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:06:40.418591 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36ede28-107f-4d87-b2d9-22d6311e9ad5 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:06:40.437470 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 735/1464] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:06:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:06:42.345509 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb409e7-18ac-47fb-b18c-eba4843ec09a None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:06:42.346101 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb409e7-18ac-47fb-b18c-eba4843ec09a None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:06:42.346101 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb409e7-18ac-47fb-b18c-eba4843ec09a None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:06:42.347445 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 730/1465] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 17:06:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 17:06:42.352632 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e8f244-88d5-48e1-9d70-1eace4e159fd None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:06:42.353080 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e8f244-88d5-48e1-9d70-1eace4e159fd None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:06:42.353184 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e8f244-88d5-48e1-9d70-1eace4e159fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:06:42.419539 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-a5e8f244-88d5-48e1-9d70-1eace4e159fd None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:06:42.420750 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-a5e8f244-88d5-48e1-9d70-1eace4e159fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:06:42.438183 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:06:42.516763 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 736/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 17:06:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:06:42.799791 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb409e7-18ac-47fb-b18c-eba4843ec09a None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:06:42.801338 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-6eb409e7-18ac-47fb-b18c-eba4843ec09a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:06:42.903331 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb409e7-18ac-47fb-b18c-eba4843ec09a None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:06:42.968681 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:06:42.975811 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbc5534-1830-430b-a4d1-1f5901d6316a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:06:42.976410 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbc5534-1830-430b-a4d1-1f5901d6316a service neutron] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:06:42.976694 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbc5534-1830-430b-a4d1-1f5901d6316a service neutron] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:06:42.976965 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbc5534-1830-430b-a4d1-1f5901d6316a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:06:42.978541 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-6bbc5534-1830-430b-a4d1-1f5901d6316a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:06:43.086856 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbc5534-1830-430b-a4d1-1f5901d6316a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:06:43.088331 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbc5534-1830-430b-a4d1-1f5901d6316a service neutron] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:06:43.106716 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 731/1467] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:06:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:06:45.047375 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb6a5a-dc25-40bb-83db-dcfc571ccd35 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:06:45.047930 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb6a5a-dc25-40bb-83db-dcfc571ccd35 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:06:45.047930 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb6a5a-dc25-40bb-83db-dcfc571ccd35 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:06:45.049260 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 737/1468] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 17:06:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:06:45.054567 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0017ff-daee-46b7-9419-c60db089149f None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:06:45.055073 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0017ff-daee-46b7-9419-c60db089149f None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:06:45.055697 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0017ff-daee-46b7-9419-c60db089149f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:06:45.120552 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-3c0017ff-daee-46b7-9419-c60db089149f None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:06:45.121761 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-3c0017ff-daee-46b7-9419-c60db089149f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:06:45.170157 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:06:45.218124 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 732/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 17:06:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:06:45.501565 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cb6a5a-dc25-40bb-83db-dcfc571ccd35 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:06:45.503107 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-b6cb6a5a-dc25-40bb-83db-dcfc571ccd35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:06:45.604794 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cb6a5a-dc25-40bb-83db-dcfc571ccd35 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:06:45.651319 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:06:45.676239 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724118c5-5806-4c0e-9fb7-db5b5527dba9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:06:45.676849 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724118c5-5806-4c0e-9fb7-db5b5527dba9 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:06:45.677122 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724118c5-5806-4c0e-9fb7-db5b5527dba9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:06:45.677389 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724118c5-5806-4c0e-9fb7-db5b5527dba9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:06:45.679811 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-724118c5-5806-4c0e-9fb7-db5b5527dba9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:06:45.786469 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724118c5-5806-4c0e-9fb7-db5b5527dba9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:06:45.787945 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724118c5-5806-4c0e-9fb7-db5b5527dba9 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:06:45.806748 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 738/1470] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:06:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:06:55.216376 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=22 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:06:55.804437 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=66 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:07:05.471659 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2020105-5e79-436b-b3a3-b20155e1968f None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:07:05.472232 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2020105-5e79-436b-b3a3-b20155e1968f None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:07:05.472232 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2020105-5e79-436b-b3a3-b20155e1968f None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:07:05.473548 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 733/1471] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 17:07:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:07:05.479082 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad200195-2021-4528-a9b3-ade060f6b5e1 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:07:05.479488 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad200195-2021-4528-a9b3-ade060f6b5e1 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:07:05.479631 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad200195-2021-4528-a9b3-ade060f6b5e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:07:05.544018 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-ad200195-2021-4528-a9b3-ade060f6b5e1 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:07:05.545209 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-ad200195-2021-4528-a9b3-ade060f6b5e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:07:05.640793 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 739/1472] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 17:07:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:07:05.946592 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd803861-57c6-4058-ae7c-0587c2f7cc8b None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:07:05.946886 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd803861-57c6-4058-ae7c-0587c2f7cc8b None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:07:05.947205 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd803861-57c6-4058-ae7c-0587c2f7cc8b None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:07:05.948529 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 734/1473] 10.222.0.82 () {54 vars in 967 bytes} [Thu Aug 18 17:07:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:07:05.953317 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31fbd68-9b71-4029-8b5a-cf406aa407ca None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:07:05.953637 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31fbd68-9b71-4029-8b5a-cf406aa407ca None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:07:05.953910 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31fbd68-9b71-4029-8b5a-cf406aa407ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:07:06.028458 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-f31fbd68-9b71-4029-8b5a-cf406aa407ca None None] MFA Rules not processed for user `2bee1ba6bb324de4915904f83349b3c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:07:06.029667 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-f31fbd68-9b71-4029-8b5a-cf406aa407ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:07:06.049631 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:07:06.104576 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 740/1474] 10.222.0.82 () {58 vars in 1064 bytes} [Thu Aug 18 17:07:05 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:07:06.112299 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd803861-57c6-4058-ae7c-0587c2f7cc8b None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:07:06.113819 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-dd803861-57c6-4058-ae7c-0587c2f7cc8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:07:06.221004 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd803861-57c6-4058-ae7c-0587c2f7cc8b None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:07:06.294053 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f391b0ba-7326-4fb5-bf9c-235b46997042 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bee1ba6bb324de4915904f83349b3c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:07:06.294654 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f391b0ba-7326-4fb5-bf9c-235b46997042 service glance] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:07:06.294954 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f391b0ba-7326-4fb5-bf9c-235b46997042 service glance] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:07:06.295229 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f391b0ba-7326-4fb5-bf9c-235b46997042 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:07:06.296801 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-f391b0ba-7326-4fb5-bf9c-235b46997042 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:07:06.350629 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:07:06.408682 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f391b0ba-7326-4fb5-bf9c-235b46997042 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:07:06.410124 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f391b0ba-7326-4fb5-bf9c-235b46997042 service glance] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:07:06.429206 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 735/1475] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:07:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:07:08.429121 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51bf8e1-257c-4fae-996c-7ee5dc7c89c1 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:07:08.429657 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51bf8e1-257c-4fae-996c-7ee5dc7c89c1 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:07:08.430114 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51bf8e1-257c-4fae-996c-7ee5dc7c89c1 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:07:08.431644 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 741/1476] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 17:07:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:07:08.436771 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010999f-b412-4ea1-b10f-e4730e40e42f None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:07:08.437196 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010999f-b412-4ea1-b10f-e4730e40e42f None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:07:08.437325 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010999f-b412-4ea1-b10f-e4730e40e42f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:07:08.502286 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-d010999f-b412-4ea1-b10f-e4730e40e42f None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:07:08.503494 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-d010999f-b412-4ea1-b10f-e4730e40e42f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:07:08.597145 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 736/1477] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 17:07:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:07:08.869654 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51bf8e1-257c-4fae-996c-7ee5dc7c89c1 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:07:08.871223 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-f51bf8e1-257c-4fae-996c-7ee5dc7c89c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:07:08.996403 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51bf8e1-257c-4fae-996c-7ee5dc7c89c1 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:07:09.044745 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:07:09.088981 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc12ee8-7d93-4e6d-bed6-cf3eee9e5f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:07:09.089584 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc12ee8-7d93-4e6d-bed6-cf3eee9e5f8e admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:07:09.089862 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc12ee8-7d93-4e6d-bed6-cf3eee9e5f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:07:09.090130 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc12ee8-7d93-4e6d-bed6-cf3eee9e5f8e admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:07:09.090761 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc12ee8-7d93-4e6d-bed6-cf3eee9e5f8e admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:07:09.092507 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: WARNING py.warnings [None req-4fc12ee8-7d93-4e6d-bed6-cf3eee9e5f8e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 18 17:07:09.092507 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: warnings.warn(msg) Aug 18 17:07:09.092507 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: Aug 18 17:07:09.093631 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc12ee8-7d93-4e6d-bed6-cf3eee9e5f8e admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:07:09.112570 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 742/1478] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 17:07:08 2022] GET /identity/v3/projects => generated 2553 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:07:11.037369 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c88dd5-a945-4dba-953c-ccef46cf30eb None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:07:11.037976 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c88dd5-a945-4dba-953c-ccef46cf30eb None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:07:11.037976 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c88dd5-a945-4dba-953c-ccef46cf30eb None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:07:11.039465 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 737/1479] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 17:07:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:07:11.044894 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c7ee3d-018b-47e8-9a8d-9c1f1c0d693e None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:07:11.045301 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c7ee3d-018b-47e8-9a8d-9c1f1c0d693e None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:07:11.045477 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c7ee3d-018b-47e8-9a8d-9c1f1c0d693e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:07:11.112791 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-79c7ee3d-018b-47e8-9a8d-9c1f1c0d693e None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:07:11.114000 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-79c7ee3d-018b-47e8-9a8d-9c1f1c0d693e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:07:11.209301 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 743/1480] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 17:07:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:07:11.505281 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c88dd5-a945-4dba-953c-ccef46cf30eb None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:07:11.507661 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-a3c88dd5-a945-4dba-953c-ccef46cf30eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:07:11.517356 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:07:11.652991 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c88dd5-a945-4dba-953c-ccef46cf30eb None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:07:11.731552 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:07:11.764360 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36777125-8e41-4bad-b54d-501205d23079 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55723c756cae4e7ab636054db48de02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:07:11.764907 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36777125-8e41-4bad-b54d-501205d23079 service nova] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:07:11.765186 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36777125-8e41-4bad-b54d-501205d23079 service nova] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:07:11.765474 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36777125-8e41-4bad-b54d-501205d23079 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:07:11.767092 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-36777125-8e41-4bad-b54d-501205d23079 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:07:11.855572 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36777125-8e41-4bad-b54d-501205d23079 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:07:11.857179 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36777125-8e41-4bad-b54d-501205d23079 service nova] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:07:11.876038 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 738/1481] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:07:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:07:13.957593 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5768c5d-7ee6-4eec-ab37-f836c07edd44 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:07:13.958118 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5768c5d-7ee6-4eec-ab37-f836c07edd44 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:07:13.958237 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5768c5d-7ee6-4eec-ab37-f836c07edd44 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:07:13.959702 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 744/1482] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 17:07:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:07:13.964996 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9712ee9d-c5ae-4671-a325-52a6555bd1d5 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:07:13.965457 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9712ee9d-c5ae-4671-a325-52a6555bd1d5 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:07:13.965561 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9712ee9d-c5ae-4671-a325-52a6555bd1d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:07:14.030936 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-9712ee9d-c5ae-4671-a325-52a6555bd1d5 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:07:14.032074 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-9712ee9d-c5ae-4671-a325-52a6555bd1d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:07:14.079866 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:07:14.130122 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 739/1483] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 17:07:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 17:07:14.417691 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5768c5d-7ee6-4eec-ab37-f836c07edd44 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:07:14.419335 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-b5768c5d-7ee6-4eec-ab37-f836c07edd44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:07:14.443741 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:07:14.521399 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5768c5d-7ee6-4eec-ab37-f836c07edd44 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:07:14.593200 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f104d56f-04b0-4bf6-8b2b-8bd75b4a2021 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:07:14.593799 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f104d56f-04b0-4bf6-8b2b-8bd75b4a2021 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:07:14.594078 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f104d56f-04b0-4bf6-8b2b-8bd75b4a2021 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:07:14.594351 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f104d56f-04b0-4bf6-8b2b-8bd75b4a2021 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:07:14.595951 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-f104d56f-04b0-4bf6-8b2b-8bd75b4a2021 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:07:14.668929 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:07:14.704506 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f104d56f-04b0-4bf6-8b2b-8bd75b4a2021 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:07:14.705925 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f104d56f-04b0-4bf6-8b2b-8bd75b4a2021 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:07:14.723602 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 745/1484] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:07:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:07:16.675995 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a455a3a2-9355-4b4f-a5af-91f9b5e080eb None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:07:16.676547 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a455a3a2-9355-4b4f-a5af-91f9b5e080eb None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:07:16.676547 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a455a3a2-9355-4b4f-a5af-91f9b5e080eb None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:07:16.678045 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 740/1485] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 17:07:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:07:16.683251 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46d4e70-03a5-48d9-805a-0a37b0ea5cda None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:07:16.683654 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46d4e70-03a5-48d9-805a-0a37b0ea5cda None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:07:16.683807 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46d4e70-03a5-48d9-805a-0a37b0ea5cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:07:16.749905 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-b46d4e70-03a5-48d9-805a-0a37b0ea5cda None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:07:16.751075 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-b46d4e70-03a5-48d9-805a-0a37b0ea5cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:07:16.848042 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 746/1486] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 17:07:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:07:17.140190 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a455a3a2-9355-4b4f-a5af-91f9b5e080eb None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:07:17.142116 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-a455a3a2-9355-4b4f-a5af-91f9b5e080eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:07:17.247683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a455a3a2-9355-4b4f-a5af-91f9b5e080eb None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:07:17.321290 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:07:17.323084 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c237152c-689d-423b-b18f-cd76a4762e6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:07:17.323697 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c237152c-689d-423b-b18f-cd76a4762e6b service neutron] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:07:17.323978 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c237152c-689d-423b-b18f-cd76a4762e6b service neutron] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:07:17.324250 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c237152c-689d-423b-b18f-cd76a4762e6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:07:17.325836 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-c237152c-689d-423b-b18f-cd76a4762e6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:07:17.437740 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c237152c-689d-423b-b18f-cd76a4762e6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:07:17.439197 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c237152c-689d-423b-b18f-cd76a4762e6b service neutron] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:07:17.458418 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 741/1487] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:07:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:07:19.471265 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6897f66b-b88f-424d-9cf6-0a57add7a13c None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:07:19.471834 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6897f66b-b88f-424d-9cf6-0a57add7a13c None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:07:19.471834 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6897f66b-b88f-424d-9cf6-0a57add7a13c None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:07:19.473186 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 747/1488] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 17:07:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:07:19.478648 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5efa28-2882-4747-97e5-11917dad41a5 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:07:19.479111 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5efa28-2882-4747-97e5-11917dad41a5 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:07:19.479292 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5efa28-2882-4747-97e5-11917dad41a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:07:19.544125 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-6f5efa28-2882-4747-97e5-11917dad41a5 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:07:19.545331 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-6f5efa28-2882-4747-97e5-11917dad41a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:07:19.598717 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:07:19.642677 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 742/1489] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 17:07:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:07:19.938280 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6897f66b-b88f-424d-9cf6-0a57add7a13c None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:07:19.939875 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-6897f66b-b88f-424d-9cf6-0a57add7a13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:07:19.976190 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:07:20.042386 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6897f66b-b88f-424d-9cf6-0a57add7a13c None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:07:20.113606 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e60a2ba-dfb8-4f3b-afc3-d07414f2c84d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:07:20.114205 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e60a2ba-dfb8-4f3b-afc3-d07414f2c84d service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:07:20.114482 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e60a2ba-dfb8-4f3b-afc3-d07414f2c84d service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:07:20.114754 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e60a2ba-dfb8-4f3b-afc3-d07414f2c84d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:07:20.116502 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-7e60a2ba-dfb8-4f3b-afc3-d07414f2c84d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:07:20.193989 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:07:20.222095 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e60a2ba-dfb8-4f3b-afc3-d07414f2c84d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:07:20.224075 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e60a2ba-dfb8-4f3b-afc3-d07414f2c84d service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:07:20.242438 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 748/1490] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:07:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:07:22.501059 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356114bc-271a-4643-8257-0d62d2cda95f None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:07:22.501644 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356114bc-271a-4643-8257-0d62d2cda95f None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:07:22.501644 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356114bc-271a-4643-8257-0d62d2cda95f None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:07:22.502963 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 743/1491] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 17:07:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:07:22.508338 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc9ea00-51a3-4d2e-befe-0a963cb34ea5 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:07:22.508728 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc9ea00-51a3-4d2e-befe-0a963cb34ea5 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:07:22.508915 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc9ea00-51a3-4d2e-befe-0a963cb34ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:07:22.574184 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-bfc9ea00-51a3-4d2e-befe-0a963cb34ea5 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:07:22.575419 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-bfc9ea00-51a3-4d2e-befe-0a963cb34ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:07:22.671864 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 749/1492] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 17:07:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:07:22.965259 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356114bc-271a-4643-8257-0d62d2cda95f None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:07:22.966819 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-356114bc-271a-4643-8257-0d62d2cda95f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:07:23.070593 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356114bc-271a-4643-8257-0d62d2cda95f None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:07:23.147368 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ac5a0a-4e38-42f6-a485-558c945a5244 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:07:23.147990 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac5a0a-4e38-42f6-a485-558c945a5244 service neutron] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:07:23.148271 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac5a0a-4e38-42f6-a485-558c945a5244 service neutron] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:07:23.148541 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac5a0a-4e38-42f6-a485-558c945a5244 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:07:23.150236 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-44ac5a0a-4e38-42f6-a485-558c945a5244 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:07:23.169820 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:07:23.291919 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ac5a0a-4e38-42f6-a485-558c945a5244 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:07:23.293372 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ac5a0a-4e38-42f6-a485-558c945a5244 service neutron] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:07:23.311481 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 744/1493] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:07:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:07:32.670085 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=82 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:07:33.309164 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=50 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:08:51.224048 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c1294d-5f6a-4818-97b8-2471a3a13b31 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:08:51.224637 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c1294d-5f6a-4818-97b8-2471a3a13b31 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:08:51.224637 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c1294d-5f6a-4818-97b8-2471a3a13b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:08:51.299896 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-b3c1294d-5f6a-4818-97b8-2471a3a13b31 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:08:51.301099 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-b3c1294d-5f6a-4818-97b8-2471a3a13b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:08:51.395684 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 750/1494] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:08:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:08:51.403987 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb99aab-1c91-481c-b140-023fc38da0e2 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:08:51.404465 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb99aab-1c91-481c-b140-023fc38da0e2 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:08:51.404567 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb99aab-1c91-481c-b140-023fc38da0e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:08:51.480108 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-5bb99aab-1c91-481c-b140-023fc38da0e2 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:08:51.481296 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-5bb99aab-1c91-481c-b140-023fc38da0e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:08:51.577556 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 745/1495] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:08:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:08:51.582864 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c1294d-5f6a-4818-97b8-2471a3a13b31 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:08:51.584093 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-b3c1294d-5f6a-4818-97b8-2471a3a13b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:08:51.596005 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c1294d-5f6a-4818-97b8-2471a3a13b31 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:08:51.655945 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:08:51.690131 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f5a1a0-5df2-4866-b07d-7ae0f71c2757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:08:51.690737 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f5a1a0-5df2-4866-b07d-7ae0f71c2757 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:08:51.691048 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f5a1a0-5df2-4866-b07d-7ae0f71c2757 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:08:51.691324 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f5a1a0-5df2-4866-b07d-7ae0f71c2757 admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:08:51.691961 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f5a1a0-5df2-4866-b07d-7ae0f71c2757 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:08:51.692263 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f5a1a0-5df2-4866-b07d-7ae0f71c2757 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:08:51.693273 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: WARNING py.warnings [None req-55f5a1a0-5df2-4866-b07d-7ae0f71c2757 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 18 17:08:51.693273 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: warnings.warn(msg) Aug 18 17:08:51.693273 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: Aug 18 17:08:51.694158 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f5a1a0-5df2-4866-b07d-7ae0f71c2757 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:08:51.706006 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 751/1496] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:08:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 17:08:51.713974 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb99aab-1c91-481c-b140-023fc38da0e2 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:08:51.724145 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb99aab-1c91-481c-b140-023fc38da0e2 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:08:51.786212 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:08:51.820210 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5fadea-7305-4723-9ebe-92e44bfd8b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:08:51.820878 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5fadea-7305-4723-9ebe-92e44bfd8b0e admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:08:51.821184 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5fadea-7305-4723-9ebe-92e44bfd8b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:08:51.821458 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5fadea-7305-4723-9ebe-92e44bfd8b0e admin admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:08:51.822084 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5fadea-7305-4723-9ebe-92e44bfd8b0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:08:51.823899 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: WARNING py.warnings [None req-fe5fadea-7305-4723-9ebe-92e44bfd8b0e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 18 17:08:51.823899 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: warnings.warn(msg) Aug 18 17:08:51.823899 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: Aug 18 17:08:51.825117 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5fadea-7305-4723-9ebe-92e44bfd8b0e admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:08:51.946234 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 746/1497] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:08:51 2022] POST /identity/v3/projects => generated 377 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:08:51.950448 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f5a1a0-5df2-4866-b07d-7ae0f71c2757 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:08:51.952005 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-55f5a1a0-5df2-4866-b07d-7ae0f71c2757 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:08:51.986291 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f5a1a0-5df2-4866-b07d-7ae0f71c2757 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:08:52.083434 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9136d7-ac35-4d39-90be-014a7b10e02c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:08:52.084056 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9136d7-ac35-4d39-90be-014a7b10e02c admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:08:52.084333 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9136d7-ac35-4d39-90be-014a7b10e02c admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:08:52.084600 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9136d7-ac35-4d39-90be-014a7b10e02c admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:08:52.085257 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9136d7-ac35-4d39-90be-014a7b10e02c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:08:52.086262 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: WARNING py.warnings [None req-7d9136d7-ac35-4d39-90be-014a7b10e02c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 18 17:08:52.086262 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: warnings.warn(msg) Aug 18 17:08:52.086262 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: Aug 18 17:08:52.087499 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9136d7-ac35-4d39-90be-014a7b10e02c admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:08:52.198620 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 752/1498] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:08:51 2022] POST /identity/v3/users => generated 289 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:08:52.203594 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5fadea-7305-4723-9ebe-92e44bfd8b0e admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:08:52.205192 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-fe5fadea-7305-4723-9ebe-92e44bfd8b0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:08:52.266860 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5fadea-7305-4723-9ebe-92e44bfd8b0e admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:08:52.350302 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:08:52.361461 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268082ce-2878-4fb1-a2b9-f268897aa507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:08:52.362071 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268082ce-2878-4fb1-a2b9-f268897aa507 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:08:52.362354 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268082ce-2878-4fb1-a2b9-f268897aa507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:08:52.362628 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268082ce-2878-4fb1-a2b9-f268897aa507 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:08:52.363304 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268082ce-2878-4fb1-a2b9-f268897aa507 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:08:52.364318 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: WARNING py.warnings [None req-268082ce-2878-4fb1-a2b9-f268897aa507 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 18 17:08:52.364318 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: warnings.warn(msg) Aug 18 17:08:52.364318 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: Aug 18 17:08:52.365230 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268082ce-2878-4fb1-a2b9-f268897aa507 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:08:52.377944 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 747/1499] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:08:52 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:08:52.382155 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9136d7-ac35-4d39-90be-014a7b10e02c admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:08:52.383719 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-7d9136d7-ac35-4d39-90be-014a7b10e02c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:08:52.395043 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9136d7-ac35-4d39-90be-014a7b10e02c admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:08:52.419164 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:08:52.489088 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9ee139-3a59-44cb-aa92-598284f4cbfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:08:52.489699 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ee139-3a59-44cb-aa92-598284f4cbfb admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:08:52.489981 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ee139-3a59-44cb-aa92-598284f4cbfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:08:52.490254 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ee139-3a59-44cb-aa92-598284f4cbfb admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:08:52.490990 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9ee139-3a59-44cb-aa92-598284f4cbfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:08:52.491973 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: WARNING py.warnings [None req-ce9ee139-3a59-44cb-aa92-598284f4cbfb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 18 17:08:52.491973 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: warnings.warn(msg) Aug 18 17:08:52.491973 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: Aug 18 17:08:52.492862 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9ee139-3a59-44cb-aa92-598284f4cbfb admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:08:52.506199 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 753/1500] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:08:52 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:08:52.510633 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268082ce-2878-4fb1-a2b9-f268897aa507 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:08:52.513902 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-268082ce-2878-4fb1-a2b9-f268897aa507 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:08:52.544900 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268082ce-2878-4fb1-a2b9-f268897aa507 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:08:52.637865 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc25eb0-6617-4b24-8d3b-c977e2973a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:08:52.638445 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc25eb0-6617-4b24-8d3b-c977e2973a77 admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:08:52.638723 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc25eb0-6617-4b24-8d3b-c977e2973a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:08:52.639141 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc25eb0-6617-4b24-8d3b-c977e2973a77 admin admin] PATH_INFO: `/v3/projects/4e916870a15a4ec58b2816559cd3b083/users/40a464734f4b45388d410083d1e3d51c/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:08:52.660667 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc25eb0-6617-4b24-8d3b-c977e2973a77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e916870a15a4ec58b2816559cd3b083, user_id=40a464734f4b45388d410083d1e3d51c, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:08:52.661693 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: WARNING py.warnings [None req-2dc25eb0-6617-4b24-8d3b-c977e2973a77 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 18 17:08:52.661693 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: warnings.warn(msg) Aug 18 17:08:52.661693 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: Aug 18 17:08:52.663657 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc25eb0-6617-4b24-8d3b-c977e2973a77 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:08:52.743705 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 748/1501] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:08:52 2022] PUT /identity/v3/projects/4e916870a15a4ec58b2816559cd3b083/users/40a464734f4b45388d410083d1e3d51c/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:08:52.756068 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825a970-3bac-4cb5-a260-813841ef0d1f None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:08:52.759973 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825a970-3bac-4cb5-a260-813841ef0d1f None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:08:52.759973 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825a970-3bac-4cb5-a260-813841ef0d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:08:52.813514 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-e825a970-3bac-4cb5-a260-813841ef0d1f None None] MFA Rules not processed for user `40a464734f4b45388d410083d1e3d51c`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:08:52.814686 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-e825a970-3bac-4cb5-a260-813841ef0d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:08:52.889796 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 754/1502] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:08:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:08:52.903345 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5701953-f87e-4f02-8124-cdc6697d0ce6 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:08:52.903639 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5701953-f87e-4f02-8124-cdc6697d0ce6 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:08:52.903949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5701953-f87e-4f02-8124-cdc6697d0ce6 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:08:52.905273 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 749/1503] 10.222.0.82 () {54 vars in 964 bytes} [Thu Aug 18 17:08:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:08:52.910561 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912cf9ee-ad98-49ee-9c37-6b050df718c9 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:08:52.910852 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912cf9ee-ad98-49ee-9c37-6b050df718c9 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:08:52.911166 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912cf9ee-ad98-49ee-9c37-6b050df718c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:08:52.941902 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:08:52.985580 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-912cf9ee-ad98-49ee-9c37-6b050df718c9 None None] MFA Rules not processed for user `85876a26a22e4c2c882883f7e978e4f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:08:52.986784 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-912cf9ee-ad98-49ee-9c37-6b050df718c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:08:53.060693 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 755/1504] 10.222.0.82 () {58 vars in 1061 bytes} [Thu Aug 18 17:08:52 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:08:53.068649 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5701953-f87e-4f02-8124-cdc6697d0ce6 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:08:53.070162 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-a5701953-f87e-4f02-8124-cdc6697d0ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:08:53.120533 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:08:53.173749 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5701953-f87e-4f02-8124-cdc6697d0ce6 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:08:53.245547 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d2b4e6-48a2-40d2-8b6a-a670db16df3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:08:53.246889 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d2b4e6-48a2-40d2-8b6a-a670db16df3a service cinder] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:08:53.247215 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d2b4e6-48a2-40d2-8b6a-a670db16df3a service cinder] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:08:53.247494 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d2b4e6-48a2-40d2-8b6a-a670db16df3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:08:53.249106 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-04d2b4e6-48a2-40d2-8b6a-a670db16df3a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:08:53.347383 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d2b4e6-48a2-40d2-8b6a-a670db16df3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:08:53.348841 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d2b4e6-48a2-40d2-8b6a-a670db16df3a service cinder] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:08:53.352321 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:08:53.368127 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 750/1505] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:08:53 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 17:08:53.413491 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6270e24-5e6a-423c-80f0-15694a6bb105 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:08:53.413895 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6270e24-5e6a-423c-80f0-15694a6bb105 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:08:53.414048 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6270e24-5e6a-423c-80f0-15694a6bb105 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:08:53.415349 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 756/1506] 10.222.0.82 () {52 vars in 830 bytes} [Thu Aug 18 17:08:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:08:53.419022 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d2b4e6-48a2-40d2-8b6a-a670db16df3a service cinder] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:08:53.420241 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-04d2b4e6-48a2-40d2-8b6a-a670db16df3a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:08:53.654663 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d2b4e6-48a2-40d2-8b6a-a670db16df3a service cinder] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:08:53.747127 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34b7d7d-b042-4cb9-ae04-796171705a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:08:53.747752 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b7d7d-b042-4cb9-ae04-796171705a54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:08:53.748027 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b7d7d-b042-4cb9-ae04-796171705a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:08:53.748295 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b7d7d-b042-4cb9-ae04-796171705a54 admin admin] PATH_INFO: `/v3/users/40a464734f4b45388d410083d1e3d51c` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:08:53.760681 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34b7d7d-b042-4cb9-ae04-796171705a54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40a464734f4b45388d410083d1e3d51c)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:08:53.761714 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: WARNING py.warnings [None req-a34b7d7d-b042-4cb9-ae04-796171705a54 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 18 17:08:53.761714 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: warnings.warn(msg) Aug 18 17:08:53.761714 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: Aug 18 17:08:53.762958 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34b7d7d-b042-4cb9-ae04-796171705a54 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:08:53.773532 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:08:54.013237 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-a34b7d7d-b042-4cb9-ae04-796171705a54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40a464734f4b45388d410083d1e3d51c'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:08:54.039225 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-a34b7d7d-b042-4cb9-ae04-796171705a54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40a464734f4b45388d410083d1e3d51c'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:08:54.039805 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-a34b7d7d-b042-4cb9-ae04-796171705a54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40a464734f4b45388d410083d1e3d51c'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:08:54.115273 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-a34b7d7d-b042-4cb9-ae04-796171705a54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40a464734f4b45388d410083d1e3d51c'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:08:54.115771 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-a34b7d7d-b042-4cb9-ae04-796171705a54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40a464734f4b45388d410083d1e3d51c'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:08:54.139507 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 751/1507] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:08:53 2022] DELETE /identity/v3/users/40a464734f4b45388d410083d1e3d51c => generated 0 bytes in 721 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:08:54.144223 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a2e28-5e82-4a59-9043-bea2ac490f84 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:08:54.144603 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a2e28-5e82-4a59-9043-bea2ac490f84 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:08:54.145014 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a2e28-5e82-4a59-9043-bea2ac490f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:08:54.203070 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-044a2e28-5e82-4a59-9043-bea2ac490f84 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:08:54.204397 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-044a2e28-5e82-4a59-9043-bea2ac490f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:08:54.283939 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:08:54.297688 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 757/1508] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:08:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:08:54.307395 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34b7d7d-b042-4cb9-ae04-796171705a54 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:08:54.308661 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-a34b7d7d-b042-4cb9-ae04-796171705a54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:08:54.412934 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34b7d7d-b042-4cb9-ae04-796171705a54 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:08:54.467248 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:08:54.485398 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101954b6-be58-4d8c-9e8d-612b3e02cf99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:08:54.486005 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101954b6-be58-4d8c-9e8d-612b3e02cf99 service neutron] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:08:54.486284 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101954b6-be58-4d8c-9e8d-612b3e02cf99 service neutron] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:08:54.486555 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101954b6-be58-4d8c-9e8d-612b3e02cf99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:08:54.488172 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-101954b6-be58-4d8c-9e8d-612b3e02cf99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:08:54.595882 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101954b6-be58-4d8c-9e8d-612b3e02cf99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:08:54.597610 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101954b6-be58-4d8c-9e8d-612b3e02cf99 service neutron] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:08:54.616156 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 752/1509] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:08:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:08:55.123070 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044a2e28-5e82-4a59-9043-bea2ac490f84 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:08:55.124308 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-044a2e28-5e82-4a59-9043-bea2ac490f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:08:55.135849 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044a2e28-5e82-4a59-9043-bea2ac490f84 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:08:55.232985 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d658d4-d03c-42e1-8159-b1a741754aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:08:55.233621 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d658d4-d03c-42e1-8159-b1a741754aba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:08:55.234083 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d658d4-d03c-42e1-8159-b1a741754aba admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:08:55.234346 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d658d4-d03c-42e1-8159-b1a741754aba admin admin] PATH_INFO: `/v3/projects/4e916870a15a4ec58b2816559cd3b083` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:08:55.244004 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d658d4-d03c-42e1-8159-b1a741754aba admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e916870a15a4ec58b2816559cd3b083)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:08:55.245022 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: WARNING py.warnings [None req-15d658d4-d03c-42e1-8159-b1a741754aba admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 18 17:08:55.245022 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: warnings.warn(msg) Aug 18 17:08:55.245022 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: Aug 18 17:08:55.246225 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d658d4-d03c-42e1-8159-b1a741754aba admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:08:55.366000 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-15d658d4-d03c-42e1-8159-b1a741754aba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e916870a15a4ec58b2816559cd3b083', 'request_id': 'req-15d658d4-d03c-42e1-8159-b1a741754aba'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:08:55.379411 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-15d658d4-d03c-42e1-8159-b1a741754aba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e916870a15a4ec58b2816559cd3b083', 'request_id': 'req-15d658d4-d03c-42e1-8159-b1a741754aba'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:08:55.480482 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-15d658d4-d03c-42e1-8159-b1a741754aba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e916870a15a4ec58b2816559cd3b083', 'request_id': 'req-15d658d4-d03c-42e1-8159-b1a741754aba'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:08:55.504601 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-15d658d4-d03c-42e1-8159-b1a741754aba admin admin] The token cache is being invalidate because project 4e916870a15a4ec58b2816559cd3b083 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:08:55.504889 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-15d658d4-d03c-42e1-8159-b1a741754aba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:08:55.505314 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-15d658d4-d03c-42e1-8159-b1a741754aba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:08:55.506711 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 758/1510] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:08:55 2022] DELETE /identity/v3/projects/4e916870a15a4ec58b2816559cd3b083 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:09:04.613878 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=62 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:09:05.502560 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:10:23.023942 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfe23e5-2e2c-49df-a844-6c8928db0536 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:10:23.024519 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfe23e5-2e2c-49df-a844-6c8928db0536 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:10:23.024519 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfe23e5-2e2c-49df-a844-6c8928db0536 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:10:23.025785 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 753/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 18 17:10:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:10:23.031082 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66112a0d-9747-4784-9265-aac35bf195f0 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:10:23.031527 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66112a0d-9747-4784-9265-aac35bf195f0 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:10:23.031636 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66112a0d-9747-4784-9265-aac35bf195f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:10:23.112434 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-66112a0d-9747-4784-9265-aac35bf195f0 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:10:23.113660 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-66112a0d-9747-4784-9265-aac35bf195f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:10:23.216335 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 759/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 18 17:10:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:10:23.373114 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfe23e5-2e2c-49df-a844-6c8928db0536 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:10:23.375306 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-0cfe23e5-2e2c-49df-a844-6c8928db0536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:10:23.492706 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfe23e5-2e2c-49df-a844-6c8928db0536 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:10:23.498172 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dbe355-ec93-47dd-b38a-511e7c29b828 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55723c756cae4e7ab636054db48de02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:10:23.498816 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dbe355-ec93-47dd-b38a-511e7c29b828 service nova] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:10:23.499133 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dbe355-ec93-47dd-b38a-511e7c29b828 service nova] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:10:23.499550 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dbe355-ec93-47dd-b38a-511e7c29b828 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:10:23.507504 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dbe355-ec93-47dd-b38a-511e7c29b828 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:10:23.509106 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dbe355-ec93-47dd-b38a-511e7c29b828 service nova] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:10:23.519266 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 754/1513] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:10:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:10:24.348939 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59138269-a698-4222-8320-be65b7973754 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:10:24.349467 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59138269-a698-4222-8320-be65b7973754 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:10:24.349467 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59138269-a698-4222-8320-be65b7973754 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:10:24.350876 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 760/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 18 17:10:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:10:24.355655 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f5384-7f27-439c-95cb-7dfa26e4b8d4 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:10:24.356082 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f5384-7f27-439c-95cb-7dfa26e4b8d4 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:10:24.356198 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f5384-7f27-439c-95cb-7dfa26e4b8d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:10:24.376470 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-851f5384-7f27-439c-95cb-7dfa26e4b8d4 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:10:24.377647 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-851f5384-7f27-439c-95cb-7dfa26e4b8d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:10:24.385392 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 755/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 18 17:10:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 17:10:24.539665 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59138269-a698-4222-8320-be65b7973754 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:10:24.554776 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59138269-a698-4222-8320-be65b7973754 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:10:24.560394 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2389be4-6ec9-4616-98ea-8ea4679ddc1a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55723c756cae4e7ab636054db48de02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:10:24.560984 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2389be4-6ec9-4616-98ea-8ea4679ddc1a service nova] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:10:24.561342 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2389be4-6ec9-4616-98ea-8ea4679ddc1a service nova] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:10:24.561842 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2389be4-6ec9-4616-98ea-8ea4679ddc1a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:10:24.569771 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2389be4-6ec9-4616-98ea-8ea4679ddc1a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:10:24.571375 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2389be4-6ec9-4616-98ea-8ea4679ddc1a service nova] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:10:24.580945 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 761/1516] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:10:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:10:25.283586 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5734082b-a322-455d-b1d5-2eeb2567a7e1 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:10:25.283868 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5734082b-a322-455d-b1d5-2eeb2567a7e1 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:10:25.284142 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5734082b-a322-455d-b1d5-2eeb2567a7e1 None None] PATH_INFO: `/v3` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:10:25.285442 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 756/1517] 10.222.0.83 () {54 vars in 925 bytes} [Thu Aug 18 17:10:25 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 17:10:25.291422 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d877e-138a-4374-afe7-91b0fce14590 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:10:25.291717 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d877e-138a-4374-afe7-91b0fce14590 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:10:25.291997 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d877e-138a-4374-afe7-91b0fce14590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:10:25.351040 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-135d877e-138a-4374-afe7-91b0fce14590 None None] MFA Rules not processed for user `29653ba1a2a74232972ef7a9000518b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:10:25.352244 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-135d877e-138a-4374-afe7-91b0fce14590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:10:25.368624 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:10:25.423247 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 762/1518] 10.222.0.83 () {58 vars in 1015 bytes} [Thu Aug 18 17:10:25 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:10:29.483723 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1269687-0749-45b8-b516-82b756045e65 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:10:29.484240 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1269687-0749-45b8-b516-82b756045e65 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:10:29.484240 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1269687-0749-45b8-b516-82b756045e65 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:10:29.485512 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 757/1519] 10.222.0.82 () {54 vars in 961 bytes} [Thu Aug 18 17:10:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:10:29.490393 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b1f0de-161e-413c-aa60-e41402308bbb None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:10:29.490826 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b1f0de-161e-413c-aa60-e41402308bbb None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:10:29.490983 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b1f0de-161e-413c-aa60-e41402308bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:10:29.511116 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-35b1f0de-161e-413c-aa60-e41402308bbb None None] MFA Rules not processed for user `29653ba1a2a74232972ef7a9000518b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:10:29.512308 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-35b1f0de-161e-413c-aa60-e41402308bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:10:29.520905 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 763/1520] 10.222.0.82 () {58 vars in 1058 bytes} [Thu Aug 18 17:10:29 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:10:29.528376 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1269687-0749-45b8-b516-82b756045e65 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:10:29.541798 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1269687-0749-45b8-b516-82b756045e65 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:10:29.547687 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8721c6-1474-44a1-a942-8c6768bad905 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29653ba1a2a74232972ef7a9000518b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:10:29.548389 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8721c6-1474-44a1-a942-8c6768bad905 service placement] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:10:29.548670 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8721c6-1474-44a1-a942-8c6768bad905 service placement] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:10:29.548939 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8721c6-1474-44a1-a942-8c6768bad905 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:10:29.557333 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8721c6-1474-44a1-a942-8c6768bad905 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:10:29.558965 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8721c6-1474-44a1-a942-8c6768bad905 service placement] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:10:29.569216 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 758/1521] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:10:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:10:35.853548 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7e7c66-fbc5-4e86-be00-8261e1e9ae3c None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:10:35.854111 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7e7c66-fbc5-4e86-be00-8261e1e9ae3c None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:10:35.854111 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7e7c66-fbc5-4e86-be00-8261e1e9ae3c None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:10:35.855497 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 764/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 18 17:10:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:10:35.860448 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9cd374-bf5f-49c7-9940-e1baad4edab6 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:10:35.860877 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9cd374-bf5f-49c7-9940-e1baad4edab6 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:10:35.860998 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9cd374-bf5f-49c7-9940-e1baad4edab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:10:35.880574 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-4c9cd374-bf5f-49c7-9940-e1baad4edab6 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:10:35.881752 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-4c9cd374-bf5f-49c7-9940-e1baad4edab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:10:35.890264 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 759/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 18 17:10:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 17:10:36.042677 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7e7c66-fbc5-4e86-be00-8261e1e9ae3c None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:10:36.066536 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7e7c66-fbc5-4e86-be00-8261e1e9ae3c None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:10:36.066536 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305c1739-1b5c-464c-b36d-a8bd4389f752 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55723c756cae4e7ab636054db48de02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:10:36.066536 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305c1739-1b5c-464c-b36d-a8bd4389f752 service nova] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:10:36.066536 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305c1739-1b5c-464c-b36d-a8bd4389f752 service nova] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:10:36.066925 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305c1739-1b5c-464c-b36d-a8bd4389f752 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:10:36.073127 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305c1739-1b5c-464c-b36d-a8bd4389f752 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:10:36.074738 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305c1739-1b5c-464c-b36d-a8bd4389f752 service nova] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:10:36.084464 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 765/1524] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:10:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:10:36.911748 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c348b2-791d-4a8f-a342-12d7bfc7fa1d None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:10:36.912274 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c348b2-791d-4a8f-a342-12d7bfc7fa1d None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:10:36.912385 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c348b2-791d-4a8f-a342-12d7bfc7fa1d None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:10:36.913651 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 760/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 18 17:10:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:10:36.918288 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc4f90-b4cb-4710-96e5-fb159b8122d9 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:10:36.918725 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc4f90-b4cb-4710-96e5-fb159b8122d9 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:10:36.918832 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc4f90-b4cb-4710-96e5-fb159b8122d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:10:36.938155 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-60cc4f90-b4cb-4710-96e5-fb159b8122d9 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:10:36.939435 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-60cc4f90-b4cb-4710-96e5-fb159b8122d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:10:36.947162 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 766/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 18 17:10:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:10:37.099076 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c348b2-791d-4a8f-a342-12d7bfc7fa1d None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:10:37.113384 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c348b2-791d-4a8f-a342-12d7bfc7fa1d None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:10:37.118766 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ded798-385d-47f2-9dab-164f14b07502 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55723c756cae4e7ab636054db48de02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:10:37.119385 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ded798-385d-47f2-9dab-164f14b07502 service nova] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:10:37.119664 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ded798-385d-47f2-9dab-164f14b07502 service nova] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:10:37.119933 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ded798-385d-47f2-9dab-164f14b07502 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:10:37.128210 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ded798-385d-47f2-9dab-164f14b07502 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:10:37.129795 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ded798-385d-47f2-9dab-164f14b07502 service nova] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:10:37.139769 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 761/1527] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:10:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:10:41.195087 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1a4b4-4d98-4dad-9818-4e5113bcd997 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:10:41.195640 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1a4b4-4d98-4dad-9818-4e5113bcd997 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:10:41.195640 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1a4b4-4d98-4dad-9818-4e5113bcd997 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:10:41.196997 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 767/1528] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 17:10:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:10:41.202605 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba993f-e73a-4666-96b4-83e70015268c None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:10:41.203040 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba993f-e73a-4666-96b4-83e70015268c None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:10:41.203195 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba993f-e73a-4666-96b4-83e70015268c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:10:41.222002 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-4fba993f-e73a-4666-96b4-83e70015268c None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:10:41.223241 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-4fba993f-e73a-4666-96b4-83e70015268c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:10:41.230916 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 762/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 17:10:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:10:41.522169 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa1a4b4-4d98-4dad-9818-4e5113bcd997 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:10:41.523920 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-caa1a4b4-4d98-4dad-9818-4e5113bcd997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:10:41.601678 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa1a4b4-4d98-4dad-9818-4e5113bcd997 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:10:41.605281 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e6c0c7-5228-4286-9820-8a5cefabb1fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:10:41.605886 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e6c0c7-5228-4286-9820-8a5cefabb1fb service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:10:41.606170 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e6c0c7-5228-4286-9820-8a5cefabb1fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:10:41.606446 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e6c0c7-5228-4286-9820-8a5cefabb1fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:10:41.615347 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e6c0c7-5228-4286-9820-8a5cefabb1fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:10:41.616779 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e6c0c7-5228-4286-9820-8a5cefabb1fb service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:10:41.626175 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 768/1530] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:10:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:10:43.564312 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da432f13-32e7-44ae-9c22-f4f052091993 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:10:43.564940 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da432f13-32e7-44ae-9c22-f4f052091993 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:10:43.564940 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da432f13-32e7-44ae-9c22-f4f052091993 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:10:43.566202 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 763/1531] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 17:10:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:10:43.571517 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ced33a-9c6b-4477-b92e-6516400c796f None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:10:43.571942 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ced33a-9c6b-4477-b92e-6516400c796f None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:10:43.572107 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ced33a-9c6b-4477-b92e-6516400c796f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:10:43.592620 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-d3ced33a-9c6b-4477-b92e-6516400c796f None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:10:43.593806 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-d3ced33a-9c6b-4477-b92e-6516400c796f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:10:43.601552 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 769/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 17:10:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:10:43.888417 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da432f13-32e7-44ae-9c22-f4f052091993 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:10:43.890109 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-da432f13-32e7-44ae-9c22-f4f052091993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:10:43.903007 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da432f13-32e7-44ae-9c22-f4f052091993 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:10:43.907093 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceace2d8-6d3c-43ae-8872-9b069a33757d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:10:43.907680 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceace2d8-6d3c-43ae-8872-9b069a33757d service neutron] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:10:43.907961 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceace2d8-6d3c-43ae-8872-9b069a33757d service neutron] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:10:43.908234 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceace2d8-6d3c-43ae-8872-9b069a33757d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:10:43.916880 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceace2d8-6d3c-43ae-8872-9b069a33757d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:10:43.918312 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceace2d8-6d3c-43ae-8872-9b069a33757d service neutron] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:10:43.928012 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 764/1533] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:10:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:10:47.500608 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af162df-9cf7-4720-bafe-a7317ac26c8d None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:10:47.501220 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af162df-9cf7-4720-bafe-a7317ac26c8d None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:10:47.501220 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af162df-9cf7-4720-bafe-a7317ac26c8d None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:10:47.502531 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 770/1534] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 17:10:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:10:47.507618 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ca948d-9a66-4030-b45f-c19bef4fd53b None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:10:47.508014 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ca948d-9a66-4030-b45f-c19bef4fd53b None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:10:47.508179 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ca948d-9a66-4030-b45f-c19bef4fd53b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:10:47.515700 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=81 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:10:47.527352 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-33ca948d-9a66-4030-b45f-c19bef4fd53b None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:10:47.528609 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-33ca948d-9a66-4030-b45f-c19bef4fd53b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:10:47.537248 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 765/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 17:10:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:10:48.084675 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af162df-9cf7-4720-bafe-a7317ac26c8d None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:10:48.086470 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-4af162df-9cf7-4720-bafe-a7317ac26c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:10:48.151959 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:10:48.165721 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af162df-9cf7-4720-bafe-a7317ac26c8d None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:10:48.169505 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc92c85-deec-468c-a60e-2f2ecde6f98d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:10:48.174236 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc92c85-deec-468c-a60e-2f2ecde6f98d service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:10:48.174236 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc92c85-deec-468c-a60e-2f2ecde6f98d service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:10:48.174236 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc92c85-deec-468c-a60e-2f2ecde6f98d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:10:48.179619 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc92c85-deec-468c-a60e-2f2ecde6f98d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:10:48.181099 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc92c85-deec-468c-a60e-2f2ecde6f98d service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:10:48.190417 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 771/1536] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:10:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:10:50.118137 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be252fa-11f1-4e64-9470-8e7e4abd0073 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:10:50.118669 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be252fa-11f1-4e64-9470-8e7e4abd0073 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:10:50.118669 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be252fa-11f1-4e64-9470-8e7e4abd0073 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:10:50.120050 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 766/1537] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 17:10:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:10:50.125258 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25944d16-e83f-4ab0-8a2a-1ec0d18f25f1 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:10:50.125669 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25944d16-e83f-4ab0-8a2a-1ec0d18f25f1 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:10:50.125809 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25944d16-e83f-4ab0-8a2a-1ec0d18f25f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:10:50.144993 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-25944d16-e83f-4ab0-8a2a-1ec0d18f25f1 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:10:50.146213 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-25944d16-e83f-4ab0-8a2a-1ec0d18f25f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:10:50.155187 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 772/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 17:10:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:10:50.720748 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be252fa-11f1-4e64-9470-8e7e4abd0073 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:10:50.722452 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-8be252fa-11f1-4e64-9470-8e7e4abd0073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:10:50.735321 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be252fa-11f1-4e64-9470-8e7e4abd0073 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:10:50.738731 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74392d70-f58b-42e8-a06f-978368fa70a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:10:50.739362 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74392d70-f58b-42e8-a06f-978368fa70a5 service cinder] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:10:50.739639 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74392d70-f58b-42e8-a06f-978368fa70a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:10:50.739906 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74392d70-f58b-42e8-a06f-978368fa70a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:10:50.748961 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74392d70-f58b-42e8-a06f-978368fa70a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:10:50.750388 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74392d70-f58b-42e8-a06f-978368fa70a5 service cinder] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:10:50.759402 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 767/1539] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:10:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:11:00.140257 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=11 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:11:00.755381 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=8 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:15.911661 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22be331-2177-422f-9083-749c279b1665 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:15.915958 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22be331-2177-422f-9083-749c279b1665 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:15.915958 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22be331-2177-422f-9083-749c279b1665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:15.951299 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-e22be331-2177-422f-9083-749c279b1665 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:15.952536 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-e22be331-2177-422f-9083-749c279b1665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:15.961494 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 773/1540] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:15.970245 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effabf12-b56c-431d-87ac-1c119a698f82 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:15.970849 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce6fbda-fbaf-447f-8b5d-cae4033ceeb3 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:15.970849 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce6fbda-fbaf-447f-8b5d-cae4033ceeb3 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:15.971198 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce6fbda-fbaf-447f-8b5d-cae4033ceeb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:15.973439 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effabf12-b56c-431d-87ac-1c119a698f82 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:15.973439 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effabf12-b56c-431d-87ac-1c119a698f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:16.001255 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-fce6fbda-fbaf-447f-8b5d-cae4033ceeb3 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:16.002612 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-fce6fbda-fbaf-447f-8b5d-cae4033ceeb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:16.006171 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-effabf12-b56c-431d-87ac-1c119a698f82 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:16.007401 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-effabf12-b56c-431d-87ac-1c119a698f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:16.016868 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 774/1541] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:16.030036 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 768/1542] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:16.030273 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56e7675-342c-4e92-a74a-d7829d15d8f8 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:16.030654 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56e7675-342c-4e92-a74a-d7829d15d8f8 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:16.030970 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56e7675-342c-4e92-a74a-d7829d15d8f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:16.034379 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effabf12-b56c-431d-87ac-1c119a698f82 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:16.051475 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effabf12-b56c-431d-87ac-1c119a698f82 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:16.061896 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ad553c-5d95-4eb4-bbdd-42e068f81c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:16.062535 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad553c-5d95-4eb4-bbdd-42e068f81c75 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:16.062817 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad553c-5d95-4eb4-bbdd-42e068f81c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:16.063124 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad553c-5d95-4eb4-bbdd-42e068f81c75 admin admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:16.063781 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ad553c-5d95-4eb4-bbdd-42e068f81c75 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:12:16.064130 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-c56e7675-342c-4e92-a74a-d7829d15d8f8 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:16.064241 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ad553c-5d95-4eb4-bbdd-42e068f81c75 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:16.065298 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: WARNING py.warnings [None req-35ad553c-5d95-4eb4-bbdd-42e068f81c75 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 18 17:12:16.065298 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: warnings.warn(msg) Aug 18 17:12:16.065298 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: Aug 18 17:12:16.065551 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-c56e7675-342c-4e92-a74a-d7829d15d8f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:16.066290 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ad553c-5d95-4eb4-bbdd-42e068f81c75 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:16.081907 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 769/1543] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:12:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 17:12:16.083227 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240f1fba-d5f2-4418-a5dc-4fa589f0eff4 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:16.083516 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240f1fba-d5f2-4418-a5dc-4fa589f0eff4 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:16.083826 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240f1fba-d5f2-4418-a5dc-4fa589f0eff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:16.089278 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 775/1544] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Aug 18 17:12:16.099063 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56e7675-342c-4e92-a74a-d7829d15d8f8 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:16.109799 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56e7675-342c-4e92-a74a-d7829d15d8f8 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:16.119827 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-240f1fba-d5f2-4418-a5dc-4fa589f0eff4 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:16.119827 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-240f1fba-d5f2-4418-a5dc-4fa589f0eff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:16.129881 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed116c24-9f84-40b1-9469-0016edf4e434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:16.130555 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed116c24-9f84-40b1-9469-0016edf4e434 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:16.130801 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed116c24-9f84-40b1-9469-0016edf4e434 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:16.131110 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed116c24-9f84-40b1-9469-0016edf4e434 admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:16.131905 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed116c24-9f84-40b1-9469-0016edf4e434 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:12:16.132221 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed116c24-9f84-40b1-9469-0016edf4e434 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:16.134537 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed116c24-9f84-40b1-9469-0016edf4e434 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:16.136686 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 770/1545] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:16.137890 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240f1fba-d5f2-4418-a5dc-4fa589f0eff4 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:16.155523 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240f1fba-d5f2-4418-a5dc-4fa589f0eff4 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:16.166716 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 776/1546] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:12:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 17:12:16.175847 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d683e1-be19-4eee-abee-5795efc95261 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:16.175847 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d683e1-be19-4eee-abee-5795efc95261 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:16.175847 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d683e1-be19-4eee-abee-5795efc95261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:16.176118 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1260bb0d-8e16-4b76-85db-4b744efbe958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:16.176118 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1260bb0d-8e16-4b76-85db-4b744efbe958 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:16.176118 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1260bb0d-8e16-4b76-85db-4b744efbe958 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:16.176118 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1260bb0d-8e16-4b76-85db-4b744efbe958 admin admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:16.176441 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1260bb0d-8e16-4b76-85db-4b744efbe958 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:16.176441 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1260bb0d-8e16-4b76-85db-4b744efbe958 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:16.218296 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-26d683e1-be19-4eee-abee-5795efc95261 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:16.222028 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-26d683e1-be19-4eee-abee-5795efc95261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:16.228985 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 777/1547] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:16.232109 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9fd8b3-3dec-41d1-b2b1-7d64b11e2e91 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:16.232458 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9fd8b3-3dec-41d1-b2b1-7d64b11e2e91 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:16.232890 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9fd8b3-3dec-41d1-b2b1-7d64b11e2e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:16.254291 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-4b9fd8b3-3dec-41d1-b2b1-7d64b11e2e91 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:16.255531 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-4b9fd8b3-3dec-41d1-b2b1-7d64b11e2e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:16.265646 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 778/1548] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:16.266265 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9fd8b3-3dec-41d1-b2b1-7d64b11e2e91 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:16.276254 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9fd8b3-3dec-41d1-b2b1-7d64b11e2e91 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:16.287377 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0506bb27-eec2-45b2-a5ad-1a34f78134cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:16.288340 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0506bb27-eec2-45b2-a5ad-1a34f78134cd admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:16.288798 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0506bb27-eec2-45b2-a5ad-1a34f78134cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:16.289232 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0506bb27-eec2-45b2-a5ad-1a34f78134cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:16.289900 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0506bb27-eec2-45b2-a5ad-1a34f78134cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:16.291002 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: WARNING py.warnings [None req-0506bb27-eec2-45b2-a5ad-1a34f78134cd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 18 17:12:16.291002 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: warnings.warn(msg) Aug 18 17:12:16.291002 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: Aug 18 17:12:16.292276 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0506bb27-eec2-45b2-a5ad-1a34f78134cd admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:16.307141 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 771/1549] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:12:16 2022] POST /identity/v3/projects => generated 381 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:12:16.308919 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba9c49-d479-47b1-854a-3fa715417523 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:16.309223 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba9c49-d479-47b1-854a-3fa715417523 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:16.309494 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba9c49-d479-47b1-854a-3fa715417523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:16.330891 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-4bba9c49-d479-47b1-854a-3fa715417523 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:16.332424 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-4bba9c49-d479-47b1-854a-3fa715417523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:16.390952 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 772/1550] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:16.392070 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 779/1551] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:12:16 2022] POST /identity/v3/projects => generated 389 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:12:16.392453 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1cbf3d-46b8-4331-9e93-5b615089fb4b None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:16.392835 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1cbf3d-46b8-4331-9e93-5b615089fb4b None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:16.393204 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1cbf3d-46b8-4331-9e93-5b615089fb4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:16.393597 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0506bb27-eec2-45b2-a5ad-1a34f78134cd admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:16.412696 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-2b1cbf3d-46b8-4331-9e93-5b615089fb4b None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:16.413879 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-2b1cbf3d-46b8-4331-9e93-5b615089fb4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:16.459723 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0506bb27-eec2-45b2-a5ad-1a34f78134cd admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:16.464661 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b894f1-f735-4152-93c6-4bf63ae085cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:16.465274 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b894f1-f735-4152-93c6-4bf63ae085cb admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:16.465559 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b894f1-f735-4152-93c6-4bf63ae085cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:16.465843 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b894f1-f735-4152-93c6-4bf63ae085cb admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:16.466481 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b894f1-f735-4152-93c6-4bf63ae085cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:12:16.466787 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b894f1-f735-4152-93c6-4bf63ae085cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:16.468425 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b894f1-f735-4152-93c6-4bf63ae085cb admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:16.473811 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 773/1552] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:16.475098 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f0a10-c52a-4754-a3da-48ab95cd34e7 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:16.475496 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f0a10-c52a-4754-a3da-48ab95cd34e7 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:16.475789 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f0a10-c52a-4754-a3da-48ab95cd34e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:16.481412 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 780/1553] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:12:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 17:12:16.482729 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2312077-fa56-4c9c-9cf3-305be419cf20 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:16.483051 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2312077-fa56-4c9c-9cf3-305be419cf20 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:16.483330 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2312077-fa56-4c9c-9cf3-305be419cf20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:16.504247 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-c2312077-fa56-4c9c-9cf3-305be419cf20 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:16.504621 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-a48f0a10-c52a-4754-a3da-48ab95cd34e7 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:16.505460 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-c2312077-fa56-4c9c-9cf3-305be419cf20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:16.505802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-a48f0a10-c52a-4754-a3da-48ab95cd34e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:16.513558 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 781/1554] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:16.514134 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 774/1555] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:16.514305 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2312077-fa56-4c9c-9cf3-305be419cf20 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:16.515205 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57535f08-12c3-450a-a5eb-441373e1b3dc None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:16.515490 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57535f08-12c3-450a-a5eb-441373e1b3dc None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:16.515761 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57535f08-12c3-450a-a5eb-441373e1b3dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:16.523540 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2312077-fa56-4c9c-9cf3-305be419cf20 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:16.528623 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bf7ccd-56cf-4ad5-8dae-90b2fb3ee7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:16.529237 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bf7ccd-56cf-4ad5-8dae-90b2fb3ee7b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:16.529518 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bf7ccd-56cf-4ad5-8dae-90b2fb3ee7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:16.529788 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bf7ccd-56cf-4ad5-8dae-90b2fb3ee7b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:16.530408 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bf7ccd-56cf-4ad5-8dae-90b2fb3ee7b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:16.532370 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bf7ccd-56cf-4ad5-8dae-90b2fb3ee7b4 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:16.536241 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-57535f08-12c3-450a-a5eb-441373e1b3dc None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:16.537495 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-57535f08-12c3-450a-a5eb-441373e1b3dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:16.545956 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 775/1556] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:16.546721 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57535f08-12c3-450a-a5eb-441373e1b3dc None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:16.559513 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57535f08-12c3-450a-a5eb-441373e1b3dc None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:16.564064 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0ee170-2721-4c70-82ce-7b22895006e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:16.564647 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ee170-2721-4c70-82ce-7b22895006e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:16.564932 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ee170-2721-4c70-82ce-7b22895006e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:16.565216 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ee170-2721-4c70-82ce-7b22895006e1 admin admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:16.565842 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0ee170-2721-4c70-82ce-7b22895006e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:16.566893 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: WARNING py.warnings [None req-8a0ee170-2721-4c70-82ce-7b22895006e1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 18 17:12:16.566893 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: warnings.warn(msg) Aug 18 17:12:16.566893 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: Aug 18 17:12:16.568423 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0ee170-2721-4c70-82ce-7b22895006e1 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:16.633620 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 782/1557] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:12:16 2022] POST /identity/v3/users => generated 291 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:12:16.634875 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced3330-078c-4abf-97b5-6751f43d76c4 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:16.635197 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced3330-078c-4abf-97b5-6751f43d76c4 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:16.635473 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced3330-078c-4abf-97b5-6751f43d76c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:16.654037 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-bced3330-078c-4abf-97b5-6751f43d76c4 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:16.655257 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-bced3330-078c-4abf-97b5-6751f43d76c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:16.663111 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 783/1558] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:16.669439 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 776/1559] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:12:16 2022] POST /identity/v3/users => generated 295 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:12:16.669572 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3702e08c-9844-40bd-ad9e-c4dcb38bc87a None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:16.669678 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0ee170-2721-4c70-82ce-7b22895006e1 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:16.669771 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3702e08c-9844-40bd-ad9e-c4dcb38bc87a None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:16.669771 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3702e08c-9844-40bd-ad9e-c4dcb38bc87a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:16.676906 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0ee170-2721-4c70-82ce-7b22895006e1 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:16.682427 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e26276-32af-4a17-a749-ded7259e019d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:16.683060 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e26276-32af-4a17-a749-ded7259e019d admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:16.683407 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e26276-32af-4a17-a749-ded7259e019d admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:16.683668 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e26276-32af-4a17-a749-ded7259e019d admin admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:16.684278 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e26276-32af-4a17-a749-ded7259e019d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:16.686198 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e26276-32af-4a17-a749-ded7259e019d admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:16.687206 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-3702e08c-9844-40bd-ad9e-c4dcb38bc87a None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:16.688392 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-3702e08c-9844-40bd-ad9e-c4dcb38bc87a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:16.696394 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 784/1560] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:16.697252 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3702e08c-9844-40bd-ad9e-c4dcb38bc87a None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:16.698435 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:16.706565 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3702e08c-9844-40bd-ad9e-c4dcb38bc87a None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:16.712018 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044596cb-5c46-488a-b4d4-fd9ac36a55c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:16.712018 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044596cb-5c46-488a-b4d4-fd9ac36a55c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:16.712266 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044596cb-5c46-488a-b4d4-fd9ac36a55c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:16.712384 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044596cb-5c46-488a-b4d4-fd9ac36a55c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:16.713021 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044596cb-5c46-488a-b4d4-fd9ac36a55c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:12:16.713358 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044596cb-5c46-488a-b4d4-fd9ac36a55c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:16.714964 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044596cb-5c46-488a-b4d4-fd9ac36a55c0 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:16.726325 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 785/1561] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:12:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 17:12:16.727984 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68186784-9f9a-4dfb-a1bf-89684bfd2639 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:16.728271 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68186784-9f9a-4dfb-a1bf-89684bfd2639 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:16.728611 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68186784-9f9a-4dfb-a1bf-89684bfd2639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:16.747522 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-68186784-9f9a-4dfb-a1bf-89684bfd2639 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:16.748717 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-68186784-9f9a-4dfb-a1bf-89684bfd2639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:16.756499 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 786/1562] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:16.757635 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260aa881-9e9b-4be5-92a4-8d25f8fbeb13 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:16.757919 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260aa881-9e9b-4be5-92a4-8d25f8fbeb13 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:16.758193 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260aa881-9e9b-4be5-92a4-8d25f8fbeb13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:16.778114 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-260aa881-9e9b-4be5-92a4-8d25f8fbeb13 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:16.779345 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-260aa881-9e9b-4be5-92a4-8d25f8fbeb13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:16.788958 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 777/1563] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:12:16 2022] POST /identity/v3/projects => generated 371 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:12:16.790239 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e26276-32af-4a17-a749-ded7259e019d admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:16.791298 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 787/1564] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:16.795702 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260aa881-9e9b-4be5-92a4-8d25f8fbeb13 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:16.806731 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260aa881-9e9b-4be5-92a4-8d25f8fbeb13 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:16.825907 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:16.853964 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e26276-32af-4a17-a749-ded7259e019d admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:16.858942 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3e748d-7909-4e28-8e56-2ea655d8111c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:16.859548 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e748d-7909-4e28-8e56-2ea655d8111c admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:16.859826 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e748d-7909-4e28-8e56-2ea655d8111c admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:16.860103 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e748d-7909-4e28-8e56-2ea655d8111c admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:16.860736 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3e748d-7909-4e28-8e56-2ea655d8111c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:16.862333 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3e748d-7909-4e28-8e56-2ea655d8111c admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:16.864315 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5be8fe6-db6f-4c28-ab91-196f629c8e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:16.864925 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5be8fe6-db6f-4c28-ab91-196f629c8e42 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:16.865204 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5be8fe6-db6f-4c28-ab91-196f629c8e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:16.865473 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5be8fe6-db6f-4c28-ab91-196f629c8e42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:16.866104 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5be8fe6-db6f-4c28-ab91-196f629c8e42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:16.867738 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5be8fe6-db6f-4c28-ab91-196f629c8e42 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:16.872279 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 778/1565] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:12:16 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:12:16.873082 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3e748d-7909-4e28-8e56-2ea655d8111c admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:16.880019 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 788/1566] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:12:16 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:12:16.891279 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5be8fe6-db6f-4c28-ab91-196f629c8e42 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:16.891403 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3e748d-7909-4e28-8e56-2ea655d8111c admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:16.891403 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0426e552-3287-43ca-9b45-0cd0084a8e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:16.891403 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426e552-3287-43ca-9b45-0cd0084a8e46 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:16.891403 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426e552-3287-43ca-9b45-0cd0084a8e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:16.891764 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426e552-3287-43ca-9b45-0cd0084a8e46 admin admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:16.891764 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0426e552-3287-43ca-9b45-0cd0084a8e46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:12:16.891764 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0426e552-3287-43ca-9b45-0cd0084a8e46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:16.891981 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5be8fe6-db6f-4c28-ab91-196f629c8e42 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:16.892063 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0426e552-3287-43ca-9b45-0cd0084a8e46 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:16.896207 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b347ff21-6794-4aa9-8850-97f6c17e6297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:16.896803 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b347ff21-6794-4aa9-8850-97f6c17e6297 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:16.897093 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b347ff21-6794-4aa9-8850-97f6c17e6297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:16.897371 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b347ff21-6794-4aa9-8850-97f6c17e6297 admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:16.898006 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b347ff21-6794-4aa9-8850-97f6c17e6297 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:12:16.898315 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b347ff21-6794-4aa9-8850-97f6c17e6297 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:16.900531 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b347ff21-6794-4aa9-8850-97f6c17e6297 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:16.904590 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 779/1567] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:12:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 17:12:16.907775 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0426e552-3287-43ca-9b45-0cd0084a8e46 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:16.912949 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 789/1568] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:12:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 17:12:16.913971 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b347ff21-6794-4aa9-8850-97f6c17e6297 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:16.923734 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0426e552-3287-43ca-9b45-0cd0084a8e46 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:16.925208 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b347ff21-6794-4aa9-8850-97f6c17e6297 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:16.929934 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c1043e-5607-429b-888f-73afde28c234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:16.930567 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c1043e-5607-429b-888f-73afde28c234 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:16.930846 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c1043e-5607-429b-888f-73afde28c234 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:16.931289 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c1043e-5607-429b-888f-73afde28c234 admin admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:16.931843 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02095d82-03fc-4211-8c4e-cd106c81bed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:16.931982 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c1043e-5607-429b-888f-73afde28c234 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:16.932467 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02095d82-03fc-4211-8c4e-cd106c81bed0 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:16.932747 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02095d82-03fc-4211-8c4e-cd106c81bed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:16.933018 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02095d82-03fc-4211-8c4e-cd106c81bed0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:16.933643 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02095d82-03fc-4211-8c4e-cd106c81bed0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:12:16.933807 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c1043e-5607-429b-888f-73afde28c234 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:16.933944 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02095d82-03fc-4211-8c4e-cd106c81bed0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:16.935545 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02095d82-03fc-4211-8c4e-cd106c81bed0 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:16.947363 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 790/1569] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:12:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 17:12:16.948540 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02095d82-03fc-4211-8c4e-cd106c81bed0 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:16.959437 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02095d82-03fc-4211-8c4e-cd106c81bed0 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:16.964796 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52335dcd-f62f-439e-90ba-bafd1c98f430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:16.965625 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52335dcd-f62f-439e-90ba-bafd1c98f430 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:16.965727 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52335dcd-f62f-439e-90ba-bafd1c98f430 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:16.965956 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52335dcd-f62f-439e-90ba-bafd1c98f430 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:16.966572 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52335dcd-f62f-439e-90ba-bafd1c98f430 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:16.968534 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52335dcd-f62f-439e-90ba-bafd1c98f430 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:17.010921 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 780/1570] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:12:16 2022] POST /identity/v3/projects => generated 381 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:12:17.010921 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c1043e-5607-429b-888f-73afde28c234 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:17.056483 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 791/1571] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:12:16 2022] POST /identity/v3/users => generated 286 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:12:17.057311 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52335dcd-f62f-439e-90ba-bafd1c98f430 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:17.069057 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52335dcd-f62f-439e-90ba-bafd1c98f430 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:17.070492 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c1043e-5607-429b-888f-73afde28c234 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:17.074279 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6e2546-612b-430d-883a-cbf75fb7ac64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:17.074869 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6e2546-612b-430d-883a-cbf75fb7ac64 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:17.075176 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6e2546-612b-430d-883a-cbf75fb7ac64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:17.075284 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0d8791-347c-46ba-bc73-61eeb457a220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:17.075447 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6e2546-612b-430d-883a-cbf75fb7ac64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:17.075894 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0d8791-347c-46ba-bc73-61eeb457a220 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:17.076072 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6e2546-612b-430d-883a-cbf75fb7ac64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:17.076177 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0d8791-347c-46ba-bc73-61eeb457a220 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:17.076433 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0d8791-347c-46ba-bc73-61eeb457a220 admin admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:17.077167 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0d8791-347c-46ba-bc73-61eeb457a220 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:12:17.077470 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0d8791-347c-46ba-bc73-61eeb457a220 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:17.077774 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6e2546-612b-430d-883a-cbf75fb7ac64 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:17.079314 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0d8791-347c-46ba-bc73-61eeb457a220 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:17.090343 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 792/1572] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:12:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:12:17.091179 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6e2546-612b-430d-883a-cbf75fb7ac64 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:17.098402 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 781/1573] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:12:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 17:12:17.099355 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0d8791-347c-46ba-bc73-61eeb457a220 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:17.102312 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6e2546-612b-430d-883a-cbf75fb7ac64 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:17.108307 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6773a1-3a7d-43b1-a388-80335c882931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:17.108903 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6773a1-3a7d-43b1-a388-80335c882931 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:17.109177 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6773a1-3a7d-43b1-a388-80335c882931 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:17.109461 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6773a1-3a7d-43b1-a388-80335c882931 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:17.109559 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0d8791-347c-46ba-bc73-61eeb457a220 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:17.110070 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6773a1-3a7d-43b1-a388-80335c882931 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:17.111683 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6773a1-3a7d-43b1-a388-80335c882931 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:17.114164 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd576f65-5c2c-477d-ae0a-164c7f1cec84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:17.114748 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd576f65-5c2c-477d-ae0a-164c7f1cec84 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:17.115059 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd576f65-5c2c-477d-ae0a-164c7f1cec84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:17.115332 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd576f65-5c2c-477d-ae0a-164c7f1cec84 admin admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:17.115938 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd576f65-5c2c-477d-ae0a-164c7f1cec84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:17.117820 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd576f65-5c2c-477d-ae0a-164c7f1cec84 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:17.122129 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 793/1574] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:12:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:12:17.122892 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6773a1-3a7d-43b1-a388-80335c882931 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:17.131981 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6773a1-3a7d-43b1-a388-80335c882931 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:17.136814 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268d9f58-0aaa-480d-9cd3-972b03b3fdcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:17.137492 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268d9f58-0aaa-480d-9cd3-972b03b3fdcf admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:17.137769 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268d9f58-0aaa-480d-9cd3-972b03b3fdcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:17.138035 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268d9f58-0aaa-480d-9cd3-972b03b3fdcf admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:17.138636 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268d9f58-0aaa-480d-9cd3-972b03b3fdcf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:17.140526 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268d9f58-0aaa-480d-9cd3-972b03b3fdcf admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:17.235353 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 782/1575] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:12:17 2022] POST /identity/v3/projects => generated 375 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:12:17.236456 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd576f65-5c2c-477d-ae0a-164c7f1cec84 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:17.247459 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 794/1576] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:12:17 2022] POST /identity/v3/projects => generated 379 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:12:17.251311 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268d9f58-0aaa-480d-9cd3-972b03b3fdcf admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:17.265040 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:17.299862 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd576f65-5c2c-477d-ae0a-164c7f1cec84 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:17.313321 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268d9f58-0aaa-480d-9cd3-972b03b3fdcf admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:17.317988 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92387312-a1fc-4917-8c3a-3f0ecc639532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:17.322875 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92387312-a1fc-4917-8c3a-3f0ecc639532 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:17.322875 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92387312-a1fc-4917-8c3a-3f0ecc639532 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:17.322875 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92387312-a1fc-4917-8c3a-3f0ecc639532 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:17.322875 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92387312-a1fc-4917-8c3a-3f0ecc639532 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:17.322875 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92387312-a1fc-4917-8c3a-3f0ecc639532 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:17.356454 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f376a722-61d8-41ec-a846-435bcf294cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:17.357068 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f376a722-61d8-41ec-a846-435bcf294cfe admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:17.357387 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f376a722-61d8-41ec-a846-435bcf294cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:17.357666 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f376a722-61d8-41ec-a846-435bcf294cfe admin admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:17.358298 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f376a722-61d8-41ec-a846-435bcf294cfe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:17.360317 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f376a722-61d8-41ec-a846-435bcf294cfe admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:17.429948 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 795/1577] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:12:17 2022] POST /identity/v3/users => generated 291 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:12:17.431499 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92387312-a1fc-4917-8c3a-3f0ecc639532 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:17.441513 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92387312-a1fc-4917-8c3a-3f0ecc639532 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:17.446429 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72106d2b-9f6e-4c3c-865a-dfac94d87162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:17.447071 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72106d2b-9f6e-4c3c-865a-dfac94d87162 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:17.447358 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72106d2b-9f6e-4c3c-865a-dfac94d87162 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:17.447689 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72106d2b-9f6e-4c3c-865a-dfac94d87162 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:17.448317 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72106d2b-9f6e-4c3c-865a-dfac94d87162 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:17.449907 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72106d2b-9f6e-4c3c-865a-dfac94d87162 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:17.459852 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 796/1578] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:12:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:12:17.461391 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72106d2b-9f6e-4c3c-865a-dfac94d87162 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:17.474773 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72106d2b-9f6e-4c3c-865a-dfac94d87162 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:17.480169 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02da5901-2fda-4dd2-94dd-f5b6b74b1ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:17.480818 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02da5901-2fda-4dd2-94dd-f5b6b74b1ded admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:17.481099 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02da5901-2fda-4dd2-94dd-f5b6b74b1ded admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:17.481377 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02da5901-2fda-4dd2-94dd-f5b6b74b1ded admin admin] PATH_INFO: `/v3/projects/befa67f784fe4ebc8d67a70870d5c37c/users/f7d69bfc621d4729b13b4381e2e3ed10/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:17.484402 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:17.488160 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 783/1579] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:12:17 2022] POST /identity/v3/projects => generated 391 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:12:17.488966 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f376a722-61d8-41ec-a846-435bcf294cfe admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:17.497258 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02da5901-2fda-4dd2-94dd-f5b6b74b1ded admin admin] RBAC: Authorizing `identity:create_grant(project_id=befa67f784fe4ebc8d67a70870d5c37c, user_id=f7d69bfc621d4729b13b4381e2e3ed10, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:17.498287 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: WARNING py.warnings [None req-02da5901-2fda-4dd2-94dd-f5b6b74b1ded admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 18 17:12:17.498287 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: warnings.warn(msg) Aug 18 17:12:17.498287 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: Aug 18 17:12:17.500255 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02da5901-2fda-4dd2-94dd-f5b6b74b1ded admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:17.551054 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f376a722-61d8-41ec-a846-435bcf294cfe admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:17.556248 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612ba509-b90e-4c25-96a8-a1028fa35a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:17.556856 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612ba509-b90e-4c25-96a8-a1028fa35a33 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:17.557139 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612ba509-b90e-4c25-96a8-a1028fa35a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:17.557417 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612ba509-b90e-4c25-96a8-a1028fa35a33 admin admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:17.558035 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612ba509-b90e-4c25-96a8-a1028fa35a33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:17.560049 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612ba509-b90e-4c25-96a8-a1028fa35a33 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:17.576330 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 797/1580] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:12:17 2022] PUT /identity/v3/projects/befa67f784fe4ebc8d67a70870d5c37c/users/f7d69bfc621d4729b13b4381e2e3ed10/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:12:17.576330 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02da5901-2fda-4dd2-94dd-f5b6b74b1ded admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:17.642502 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02da5901-2fda-4dd2-94dd-f5b6b74b1ded admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:17.647248 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a48307d-4fa0-4497-a143-0df440193674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:17.647906 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a48307d-4fa0-4497-a143-0df440193674 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:17.648193 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a48307d-4fa0-4497-a143-0df440193674 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:17.648470 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a48307d-4fa0-4497-a143-0df440193674 admin admin] PATH_INFO: `/v3/projects/d1d1a4ac5307416f8569267bf2f2d827/users/22e737cba90b4d8890ecd692e6456470/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:17.658210 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 784/1581] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:12:17 2022] POST /identity/v3/projects => generated 385 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:12:17.663520 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612ba509-b90e-4c25-96a8-a1028fa35a33 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:17.663763 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a48307d-4fa0-4497-a143-0df440193674 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1d1a4ac5307416f8569267bf2f2d827, user_id=22e737cba90b4d8890ecd692e6456470, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:17.667294 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a48307d-4fa0-4497-a143-0df440193674 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:17.703122 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:17.720744 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612ba509-b90e-4c25-96a8-a1028fa35a33 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:17.726722 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387765e1-0669-46de-94a8-ea62f98d4d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:17.726722 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387765e1-0669-46de-94a8-ea62f98d4d9b admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:17.726722 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387765e1-0669-46de-94a8-ea62f98d4d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:17.726722 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387765e1-0669-46de-94a8-ea62f98d4d9b admin admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:17.727240 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387765e1-0669-46de-94a8-ea62f98d4d9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:17.729810 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387765e1-0669-46de-94a8-ea62f98d4d9b admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:17.748125 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 798/1582] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:12:17 2022] PUT /identity/v3/projects/d1d1a4ac5307416f8569267bf2f2d827/users/22e737cba90b4d8890ecd692e6456470/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:12:17.751888 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a48307d-4fa0-4497-a143-0df440193674 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:17.810983 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 785/1583] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:12:17 2022] POST /identity/v3/users => generated 288 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:12:17.814183 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387765e1-0669-46de-94a8-ea62f98d4d9b admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:17.814292 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a48307d-4fa0-4497-a143-0df440193674 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:17.817166 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c492d069-f1c1-4fc7-978b-25945149a7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:17.817775 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492d069-f1c1-4fc7-978b-25945149a7c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:17.818054 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492d069-f1c1-4fc7-978b-25945149a7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:17.818326 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492d069-f1c1-4fc7-978b-25945149a7c6 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:17.818965 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c492d069-f1c1-4fc7-978b-25945149a7c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:17.823188 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c492d069-f1c1-4fc7-978b-25945149a7c6 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:17.823324 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387765e1-0669-46de-94a8-ea62f98d4d9b admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:17.827997 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e490eeb1-7b9e-471c-b77f-6cab4478af0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:17.828674 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e490eeb1-7b9e-471c-b77f-6cab4478af0e admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:17.829178 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e490eeb1-7b9e-471c-b77f-6cab4478af0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:17.829453 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e490eeb1-7b9e-471c-b77f-6cab4478af0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:17.830095 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e490eeb1-7b9e-471c-b77f-6cab4478af0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:17.831747 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e490eeb1-7b9e-471c-b77f-6cab4478af0e admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:17.842476 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 786/1584] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:12:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:12:17.843301 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e490eeb1-7b9e-471c-b77f-6cab4478af0e admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:17.853026 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e490eeb1-7b9e-471c-b77f-6cab4478af0e admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:17.857570 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b891b1a4-59a4-4467-a90b-899a31b21180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:17.858139 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b891b1a4-59a4-4467-a90b-899a31b21180 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:17.858419 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b891b1a4-59a4-4467-a90b-899a31b21180 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:17.858754 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b891b1a4-59a4-4467-a90b-899a31b21180 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:17.859330 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b891b1a4-59a4-4467-a90b-899a31b21180 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:17.860892 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b891b1a4-59a4-4467-a90b-899a31b21180 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:17.871245 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 787/1585] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:12:17 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:12:17.874138 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b891b1a4-59a4-4467-a90b-899a31b21180 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:17.881202 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b891b1a4-59a4-4467-a90b-899a31b21180 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:17.885824 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14523e73-f522-4e56-be6a-75a7c370fe38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:17.886371 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14523e73-f522-4e56-be6a-75a7c370fe38 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:17.886646 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14523e73-f522-4e56-be6a-75a7c370fe38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:17.886944 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14523e73-f522-4e56-be6a-75a7c370fe38 admin admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:17.887592 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14523e73-f522-4e56-be6a-75a7c370fe38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:17.889434 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14523e73-f522-4e56-be6a-75a7c370fe38 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:17.897956 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 799/1586] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:12:17 2022] POST /identity/v3/users => generated 290 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:12:17.900225 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4416c1d-0c3f-4613-8723-33766153f61c None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:17.900225 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4416c1d-0c3f-4613-8723-33766153f61c None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:17.900457 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4416c1d-0c3f-4613-8723-33766153f61c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:17.928103 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-a4416c1d-0c3f-4613-8723-33766153f61c None None] MFA Rules not processed for user `f7d69bfc621d4729b13b4381e2e3ed10`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:17.929308 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-a4416c1d-0c3f-4613-8723-33766153f61c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:17.974324 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:17.982049 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 788/1587] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:12:17 2022] POST /identity/v3/users => generated 296 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:12:17.982853 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14523e73-f522-4e56-be6a-75a7c370fe38 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:17.994569 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14523e73-f522-4e56-be6a-75a7c370fe38 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:17.998851 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 800/1588] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:17 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:17.998851 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff11e38-374a-4f31-85f5-fe4777955a09 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:17.998851 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff11e38-374a-4f31-85f5-fe4777955a09 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:17.998851 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff11e38-374a-4f31-85f5-fe4777955a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:17.999266 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f83fa3-3f84-496c-9b79-2cbcdac3d594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:17.999763 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f83fa3-3f84-496c-9b79-2cbcdac3d594 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:18.000040 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f83fa3-3f84-496c-9b79-2cbcdac3d594 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:18.000310 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f83fa3-3f84-496c-9b79-2cbcdac3d594 admin admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:18.000917 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f83fa3-3f84-496c-9b79-2cbcdac3d594 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:18.002932 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f83fa3-3f84-496c-9b79-2cbcdac3d594 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:18.016582 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-8ff11e38-374a-4f31-85f5-fe4777955a09 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:18.017796 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-8ff11e38-374a-4f31-85f5-fe4777955a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:18.026366 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 801/1589] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:18.027218 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff11e38-374a-4f31-85f5-fe4777955a09 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:18.037204 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff11e38-374a-4f31-85f5-fe4777955a09 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:18.041940 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8722b62-54ad-41d3-8835-6d74d02d6822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:18.042533 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8722b62-54ad-41d3-8835-6d74d02d6822 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:18.042813 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8722b62-54ad-41d3-8835-6d74d02d6822 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:18.043116 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8722b62-54ad-41d3-8835-6d74d02d6822 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:18.043773 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8722b62-54ad-41d3-8835-6d74d02d6822 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:18.045351 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8722b62-54ad-41d3-8835-6d74d02d6822 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:18.055279 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 802/1590] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:12:18 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:12:18.060915 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8722b62-54ad-41d3-8835-6d74d02d6822 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:18.065295 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8722b62-54ad-41d3-8835-6d74d02d6822 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:18.069832 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293c2a33-49e5-442b-972d-72308bdab099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:18.070381 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293c2a33-49e5-442b-972d-72308bdab099 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:18.070660 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293c2a33-49e5-442b-972d-72308bdab099 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:18.071592 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293c2a33-49e5-442b-972d-72308bdab099 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:18.072191 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293c2a33-49e5-442b-972d-72308bdab099 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:18.073705 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293c2a33-49e5-442b-972d-72308bdab099 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:18.091105 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 789/1591] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:12:17 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:12:18.091105 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f83fa3-3f84-496c-9b79-2cbcdac3d594 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:18.091750 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 803/1592] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:12:18 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:12:18.091750 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293c2a33-49e5-442b-972d-72308bdab099 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:18.095974 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f83fa3-3f84-496c-9b79-2cbcdac3d594 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:18.097482 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293c2a33-49e5-442b-972d-72308bdab099 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:18.100574 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b18e7d-70a2-4b9e-9db7-9d64f1921c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:18.101228 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b18e7d-70a2-4b9e-9db7-9d64f1921c76 admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:18.101508 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b18e7d-70a2-4b9e-9db7-9d64f1921c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:18.101786 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b18e7d-70a2-4b9e-9db7-9d64f1921c76 admin admin] PATH_INFO: `/v3/projects/0534f4dae3d54429b998f82353578457/users/67673f940658480983950ba342dc77ae/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:18.102201 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88feb779-a101-4b60-9dda-3f2e952a9bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:18.103028 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88feb779-a101-4b60-9dda-3f2e952a9bec admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:18.103313 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88feb779-a101-4b60-9dda-3f2e952a9bec admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:18.103589 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88feb779-a101-4b60-9dda-3f2e952a9bec admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:18.104222 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88feb779-a101-4b60-9dda-3f2e952a9bec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:18.105820 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88feb779-a101-4b60-9dda-3f2e952a9bec admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:18.116894 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 804/1593] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:12:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:12:18.117751 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88feb779-a101-4b60-9dda-3f2e952a9bec admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:18.120599 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b18e7d-70a2-4b9e-9db7-9d64f1921c76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0534f4dae3d54429b998f82353578457, user_id=67673f940658480983950ba342dc77ae, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:18.123428 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b18e7d-70a2-4b9e-9db7-9d64f1921c76 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:18.128980 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88feb779-a101-4b60-9dda-3f2e952a9bec admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:18.134125 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a247c54-d255-4670-89c4-4eab3519956c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:18.134691 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a247c54-d255-4670-89c4-4eab3519956c admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:18.135002 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a247c54-d255-4670-89c4-4eab3519956c admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:18.135278 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a247c54-d255-4670-89c4-4eab3519956c admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:18.135907 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a247c54-d255-4670-89c4-4eab3519956c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:18.137476 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a247c54-d255-4670-89c4-4eab3519956c admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:18.147147 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 805/1594] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:12:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:12:18.147891 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a247c54-d255-4670-89c4-4eab3519956c admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:18.157923 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a247c54-d255-4670-89c4-4eab3519956c admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:18.162486 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4def4a0b-822f-4400-9ad3-a74e1fcd1680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:18.163125 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4def4a0b-822f-4400-9ad3-a74e1fcd1680 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:18.163411 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4def4a0b-822f-4400-9ad3-a74e1fcd1680 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:18.163684 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4def4a0b-822f-4400-9ad3-a74e1fcd1680 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:18.164278 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4def4a0b-822f-4400-9ad3-a74e1fcd1680 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:18.166274 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4def4a0b-822f-4400-9ad3-a74e1fcd1680 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:18.204771 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 790/1595] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:12:18 2022] PUT /identity/v3/projects/0534f4dae3d54429b998f82353578457/users/67673f940658480983950ba342dc77ae/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:12:18.206040 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b18e7d-70a2-4b9e-9db7-9d64f1921c76 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:18.260935 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b18e7d-70a2-4b9e-9db7-9d64f1921c76 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:18.264599 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a16018c-6d53-4a42-8425-76c3cbaae83e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:18.265197 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a16018c-6d53-4a42-8425-76c3cbaae83e service neutron] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:18.265475 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a16018c-6d53-4a42-8425-76c3cbaae83e service neutron] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:18.265889 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a16018c-6d53-4a42-8425-76c3cbaae83e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:18.287372 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 806/1596] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:12:18 2022] POST /identity/v3/projects => generated 381 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:12:18.288281 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4def4a0b-822f-4400-9ad3-a74e1fcd1680 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:18.329549 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a16018c-6d53-4a42-8425-76c3cbaae83e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:18.331036 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a16018c-6d53-4a42-8425-76c3cbaae83e service neutron] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:18.336161 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:18.342141 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 791/1597] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:12:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:12:18.342864 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a16018c-6d53-4a42-8425-76c3cbaae83e service neutron] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:18.352238 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4def4a0b-822f-4400-9ad3-a74e1fcd1680 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:18.357583 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a16018c-6d53-4a42-8425-76c3cbaae83e service neutron] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:18.357710 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5231dd-764d-4b23-81da-99125c257201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:18.357823 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d0ca1-fada-498d-a111-2ad2bf7518cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:18.357921 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5231dd-764d-4b23-81da-99125c257201 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:18.358008 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d0ca1-fada-498d-a111-2ad2bf7518cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:18.358091 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5231dd-764d-4b23-81da-99125c257201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:18.358178 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d0ca1-fada-498d-a111-2ad2bf7518cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:18.358252 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5231dd-764d-4b23-81da-99125c257201 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:18.358330 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d0ca1-fada-498d-a111-2ad2bf7518cb admin admin] PATH_INFO: `/v3/projects/222d54f2729340c1beb7544f29288c3c/users/3ebc6eeaa4a2479e9272298a4510b509/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:18.358719 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5231dd-764d-4b23-81da-99125c257201 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:18.360615 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5231dd-764d-4b23-81da-99125c257201 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:18.371939 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 807/1598] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:12:18 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:12:18.372849 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5231dd-764d-4b23-81da-99125c257201 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:18.378507 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15d0ca1-fada-498d-a111-2ad2bf7518cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=222d54f2729340c1beb7544f29288c3c, user_id=3ebc6eeaa4a2479e9272298a4510b509, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:18.381262 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15d0ca1-fada-498d-a111-2ad2bf7518cb admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:18.382797 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5231dd-764d-4b23-81da-99125c257201 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:18.388266 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea69c5cd-453d-4dcf-bd81-4fb6df350e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:18.388918 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea69c5cd-453d-4dcf-bd81-4fb6df350e15 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:18.389198 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea69c5cd-453d-4dcf-bd81-4fb6df350e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:18.389520 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea69c5cd-453d-4dcf-bd81-4fb6df350e15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:18.390152 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea69c5cd-453d-4dcf-bd81-4fb6df350e15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:18.391798 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea69c5cd-453d-4dcf-bd81-4fb6df350e15 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:18.401898 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 808/1599] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:12:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:12:18.402682 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea69c5cd-453d-4dcf-bd81-4fb6df350e15 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:18.412080 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea69c5cd-453d-4dcf-bd81-4fb6df350e15 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:18.416935 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3507191-cc2e-46b7-9ab0-8ddc29594d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:18.417490 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3507191-cc2e-46b7-9ab0-8ddc29594d23 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:18.417770 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3507191-cc2e-46b7-9ab0-8ddc29594d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:18.418042 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3507191-cc2e-46b7-9ab0-8ddc29594d23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:18.418638 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3507191-cc2e-46b7-9ab0-8ddc29594d23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:18.420267 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3507191-cc2e-46b7-9ab0-8ddc29594d23 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:18.431247 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 809/1600] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:12:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:12:18.434533 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3507191-cc2e-46b7-9ab0-8ddc29594d23 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:18.440954 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3507191-cc2e-46b7-9ab0-8ddc29594d23 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:18.445607 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acec7b96-0b9b-41b3-baa8-fbb3885aea16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:18.446793 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec7b96-0b9b-41b3-baa8-fbb3885aea16 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:18.447104 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec7b96-0b9b-41b3-baa8-fbb3885aea16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:18.447377 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec7b96-0b9b-41b3-baa8-fbb3885aea16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:18.447974 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acec7b96-0b9b-41b3-baa8-fbb3885aea16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:18.449503 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acec7b96-0b9b-41b3-baa8-fbb3885aea16 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:18.459202 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 810/1601] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:12:18 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:12:18.471443 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbdd652-71e4-4590-9f43-f4e6e4750f5e None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:18.471443 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbdd652-71e4-4590-9f43-f4e6e4750f5e None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:18.471443 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbdd652-71e4-4590-9f43-f4e6e4750f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:18.491413 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-8fbdd652-71e4-4590-9f43-f4e6e4750f5e None None] MFA Rules not processed for user `67673f940658480983950ba342dc77ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:18.493520 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-8fbdd652-71e4-4590-9f43-f4e6e4750f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:18.505417 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 792/1602] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:12:18 2022] PUT /identity/v3/projects/222d54f2729340c1beb7544f29288c3c/users/3ebc6eeaa4a2479e9272298a4510b509/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:12:18.506394 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d0ca1-fada-498d-a111-2ad2bf7518cb admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:18.524233 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:18.564226 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 811/1603] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:18 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:18.565052 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbdd652-71e4-4590-9f43-f4e6e4750f5e None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:18.574783 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d0ca1-fada-498d-a111-2ad2bf7518cb admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:18.576771 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbdd652-71e4-4590-9f43-f4e6e4750f5e None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:18.580120 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba6c3df-ead1-47c6-ad1f-eb772d791b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:18.580753 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba6c3df-ead1-47c6-ad1f-eb772d791b87 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:18.581031 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba6c3df-ead1-47c6-ad1f-eb772d791b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:18.581245 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7993c641-e8a5-4744-8558-f71bb5a38293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:18.581355 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba6c3df-ead1-47c6-ad1f-eb772d791b87 admin admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:18.581898 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7993c641-e8a5-4744-8558-f71bb5a38293 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:18.581999 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba6c3df-ead1-47c6-ad1f-eb772d791b87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:18.582370 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7993c641-e8a5-4744-8558-f71bb5a38293 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:18.582720 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7993c641-e8a5-4744-8558-f71bb5a38293 admin admin] PATH_INFO: `/v3/projects/6299622d253a4d149d68b4bafdece4cf/users/29192b00b5c6486b942b49c5fb2641ce/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:18.584144 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba6c3df-ead1-47c6-ad1f-eb772d791b87 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:18.599840 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7993c641-e8a5-4744-8558-f71bb5a38293 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6299622d253a4d149d68b4bafdece4cf, user_id=29192b00b5c6486b942b49c5fb2641ce, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:18.602453 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7993c641-e8a5-4744-8558-f71bb5a38293 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:18.704517 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 793/1604] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:12:18 2022] POST /identity/v3/users => generated 297 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:12:18.710345 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba6c3df-ead1-47c6-ad1f-eb772d791b87 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:18.714870 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba6c3df-ead1-47c6-ad1f-eb772d791b87 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:18.719481 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ecdf51-7f6d-4027-abcb-841cd9c0b12e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:18.720081 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ecdf51-7f6d-4027-abcb-841cd9c0b12e admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:18.720357 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ecdf51-7f6d-4027-abcb-841cd9c0b12e admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:18.720634 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ecdf51-7f6d-4027-abcb-841cd9c0b12e admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:18.721293 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ecdf51-7f6d-4027-abcb-841cd9c0b12e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:18.722876 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ecdf51-7f6d-4027-abcb-841cd9c0b12e admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:18.734478 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 812/1605] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:12:18 2022] PUT /identity/v3/projects/6299622d253a4d149d68b4bafdece4cf/users/29192b00b5c6486b942b49c5fb2641ce/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:12:18.735803 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7993c641-e8a5-4744-8558-f71bb5a38293 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:18.746830 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 794/1606] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:12:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:12:18.746830 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ecdf51-7f6d-4027-abcb-841cd9c0b12e admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:18.751044 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ecdf51-7f6d-4027-abcb-841cd9c0b12e admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:18.812794 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7993c641-e8a5-4744-8558-f71bb5a38293 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:18.814407 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d9f72c-7fed-44d7-ba9e-fe33e3d3c0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:18.815117 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d9f72c-7fed-44d7-ba9e-fe33e3d3c0f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:18.815402 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d9f72c-7fed-44d7-ba9e-fe33e3d3c0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:18.815678 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d9f72c-7fed-44d7-ba9e-fe33e3d3c0f9 admin admin] PATH_INFO: `/v3/projects/289886000851431da2e3aebc3b47be8a/users/ecdea0572cc64ca394cd4d977f536d5a/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:18.817630 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55657682-aa45-4d0b-8272-ffe8ea0c1ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:18.818295 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55657682-aa45-4d0b-8272-ffe8ea0c1ae4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:18.818564 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55657682-aa45-4d0b-8272-ffe8ea0c1ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:18.818837 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55657682-aa45-4d0b-8272-ffe8ea0c1ae4 admin admin] PATH_INFO: `/v3/projects/66c908d398424970b13440ee7f6048d6/users/e72104f038d24821be824a45e108d1ac/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:18.830316 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d9f72c-7fed-44d7-ba9e-fe33e3d3c0f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=289886000851431da2e3aebc3b47be8a, user_id=ecdea0572cc64ca394cd4d977f536d5a, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:18.833063 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d9f72c-7fed-44d7-ba9e-fe33e3d3c0f9 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:18.834200 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55657682-aa45-4d0b-8272-ffe8ea0c1ae4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66c908d398424970b13440ee7f6048d6, user_id=e72104f038d24821be824a45e108d1ac, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:18.838077 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55657682-aa45-4d0b-8272-ffe8ea0c1ae4 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:18.904510 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 795/1607] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:12:18 2022] PUT /identity/v3/projects/289886000851431da2e3aebc3b47be8a/users/ecdea0572cc64ca394cd4d977f536d5a/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:12:18.905856 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c99cd-2974-4f75-91ea-a495abcbeab3 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:18.908864 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c99cd-2974-4f75-91ea-a495abcbeab3 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:18.908864 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c99cd-2974-4f75-91ea-a495abcbeab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:18.939528 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-5f3c99cd-2974-4f75-91ea-a495abcbeab3 None None] MFA Rules not processed for user `3ebc6eeaa4a2479e9272298a4510b509`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:18.944123 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 813/1608] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:12:18 2022] PUT /identity/v3/projects/66c908d398424970b13440ee7f6048d6/users/e72104f038d24821be824a45e108d1ac/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:12:18.945728 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-5f3c99cd-2974-4f75-91ea-a495abcbeab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:18.945858 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd77899-02d8-4024-bebe-ac14e8ab7a30 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:18.945858 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd77899-02d8-4024-bebe-ac14e8ab7a30 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:18.945858 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd77899-02d8-4024-bebe-ac14e8ab7a30 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:18.951325 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 814/1609] 10.222.0.82 () {54 vars in 962 bytes} [Thu Aug 18 17:12:18 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:12:18.952374 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd77899-02d8-4024-bebe-ac14e8ab7a30 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:18.976785 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:19.018252 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 796/1610] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:18 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:19.019126 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3c99cd-2974-4f75-91ea-a495abcbeab3 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:19.029850 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd77899-02d8-4024-bebe-ac14e8ab7a30 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:19.031829 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3c99cd-2974-4f75-91ea-a495abcbeab3 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:19.036692 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c6959c-ee49-405d-984b-1f0cdf75326f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:19.036692 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c6959c-ee49-405d-984b-1f0cdf75326f admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:19.036692 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c6959c-ee49-405d-984b-1f0cdf75326f admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:19.036692 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c6959c-ee49-405d-984b-1f0cdf75326f admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:19.043584 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c6959c-ee49-405d-984b-1f0cdf75326f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:19.043584 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c6959c-ee49-405d-984b-1f0cdf75326f admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:19.043787 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfee457e-fdc0-43bd-ab3a-7bf8bac59c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:19.043787 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee457e-fdc0-43bd-ab3a-7bf8bac59c86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:19.043787 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee457e-fdc0-43bd-ab3a-7bf8bac59c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:19.043787 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee457e-fdc0-43bd-ab3a-7bf8bac59c86 admin admin] PATH_INFO: `/v3/projects/07918f455d114a719fd4b7742348e1b9/users/1db2791b42554d468e3bae04b0253ace/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:19.053346 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 815/1611] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:12:18 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:12:19.054740 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85a5e8-f168-4f36-a141-4f26c920f864 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:19.055116 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85a5e8-f168-4f36-a141-4f26c920f864 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:19.055950 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85a5e8-f168-4f36-a141-4f26c920f864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:19.062497 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfee457e-fdc0-43bd-ab3a-7bf8bac59c86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07918f455d114a719fd4b7742348e1b9, user_id=1db2791b42554d468e3bae04b0253ace, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:19.066828 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfee457e-fdc0-43bd-ab3a-7bf8bac59c86 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:19.084386 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-3b85a5e8-f168-4f36-a141-4f26c920f864 None None] MFA Rules not processed for user `29192b00b5c6486b942b49c5fb2641ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:19.085838 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-3b85a5e8-f168-4f36-a141-4f26c920f864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:19.096427 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:19.154495 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 816/1612] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:19.156163 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d5c906-1ca5-4ebf-aa07-3f57ac95e6a2 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:19.156632 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d5c906-1ca5-4ebf-aa07-3f57ac95e6a2 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:19.157315 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d5c906-1ca5-4ebf-aa07-3f57ac95e6a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:19.180971 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 797/1613] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:12:19 2022] PUT /identity/v3/projects/07918f455d114a719fd4b7742348e1b9/users/1db2791b42554d468e3bae04b0253ace/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:12:19.182513 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09835f90-ab0c-4235-a138-e2a4f009d3bb None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:19.182802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09835f90-ab0c-4235-a138-e2a4f009d3bb None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:19.184422 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09835f90-ab0c-4235-a138-e2a4f009d3bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:19.189975 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-62d5c906-1ca5-4ebf-aa07-3f57ac95e6a2 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:19.192010 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-62d5c906-1ca5-4ebf-aa07-3f57ac95e6a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:19.208062 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-09835f90-ab0c-4235-a138-e2a4f009d3bb None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:19.209610 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-09835f90-ab0c-4235-a138-e2a4f009d3bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:19.255761 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 817/1614] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:19.257018 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ee8c62-5224-48e2-b9eb-86325f4bb81a None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:19.257306 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ee8c62-5224-48e2-b9eb-86325f4bb81a None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:19.257584 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ee8c62-5224-48e2-b9eb-86325f4bb81a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:19.273011 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 798/1615] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:19.275189 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09835f90-ab0c-4235-a138-e2a4f009d3bb None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:19.287639 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09835f90-ab0c-4235-a138-e2a4f009d3bb None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:19.292913 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e386f29d-5305-4705-83c7-1de8f697ed41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:19.293589 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e386f29d-5305-4705-83c7-1de8f697ed41 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:19.293878 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e386f29d-5305-4705-83c7-1de8f697ed41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:19.294149 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e386f29d-5305-4705-83c7-1de8f697ed41 admin admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:19.294988 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e386f29d-5305-4705-83c7-1de8f697ed41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:19.297001 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e386f29d-5305-4705-83c7-1de8f697ed41 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:19.309263 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-b5ee8c62-5224-48e2-b9eb-86325f4bb81a None None] MFA Rules not processed for user `55723c756cae4e7ab636054db48de02d`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:19.310692 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-b5ee8c62-5224-48e2-b9eb-86325f4bb81a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:19.380071 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 818/1616] 10.222.0.82 () {58 vars in 1059 bytes} [Thu Aug 18 17:12:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:19.380906 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ee8c62-5224-48e2-b9eb-86325f4bb81a None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:19.391646 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ee8c62-5224-48e2-b9eb-86325f4bb81a None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:19.395543 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 799/1617] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:12:19 2022] POST /identity/v3/projects => generated 381 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:12:19.395543 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e8bff-781c-4e6c-a644-c5d08d14e435 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:19.395543 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e8bff-781c-4e6c-a644-c5d08d14e435 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:19.395543 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e8bff-781c-4e6c-a644-c5d08d14e435 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:19.395889 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 800/1618] 10.222.0.82 () {54 vars in 962 bytes} [Thu Aug 18 17:12:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:12:19.396794 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104f9cf7-550e-4fbc-beac-a9c8f8f628ac None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:19.397072 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104f9cf7-550e-4fbc-beac-a9c8f8f628ac None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:19.397345 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104f9cf7-550e-4fbc-beac-a9c8f8f628ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:19.425306 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-104f9cf7-550e-4fbc-beac-a9c8f8f628ac None None] MFA Rules not processed for user `1db2791b42554d468e3bae04b0253ace`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:19.426430 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-104f9cf7-550e-4fbc-beac-a9c8f8f628ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:19.456459 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edf0b0b-423b-4071-9f25-59161c47f0e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:19.457137 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf0b0b-423b-4071-9f25-59161c47f0e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:19.457420 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf0b0b-423b-4071-9f25-59161c47f0e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:19.458089 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf0b0b-423b-4071-9f25-59161c47f0e6 admin admin] PATH_INFO: `/v3/projects/3ad3ae716b75468cae4b02494944e0e2/users/0a066d988e4648e0847a8bcd406ddfe5/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:19.473544 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edf0b0b-423b-4071-9f25-59161c47f0e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ad3ae716b75468cae4b02494944e0e2, user_id=0a066d988e4648e0847a8bcd406ddfe5, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:19.476387 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edf0b0b-423b-4071-9f25-59161c47f0e6 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:19.496679 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 801/1619] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:19.496679 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104f9cf7-550e-4fbc-beac-a9c8f8f628ac None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:19.506162 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:19.550397 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:19.554751 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104f9cf7-550e-4fbc-beac-a9c8f8f628ac None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:19.559195 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 819/1620] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:12:19 2022] PUT /identity/v3/projects/3ad3ae716b75468cae4b02494944e0e2/users/0a066d988e4648e0847a8bcd406ddfe5/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:12:19.560075 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edf0b0b-423b-4071-9f25-59161c47f0e6 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:19.615677 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc07b92-3ba8-4a14-ac40-f9c80ba97801 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:19.617520 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc07b92-3ba8-4a14-ac40-f9c80ba97801 service neutron] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:19.618831 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc07b92-3ba8-4a14-ac40-f9c80ba97801 service neutron] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:19.621923 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc07b92-3ba8-4a14-ac40-f9c80ba97801 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:19.633389 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edf0b0b-423b-4071-9f25-59161c47f0e6 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:19.638824 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c480851a-7f5c-453a-a938-fc298f5fa022 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:19.639987 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c480851a-7f5c-453a-a938-fc298f5fa022 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:19.640585 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c480851a-7f5c-453a-a938-fc298f5fa022 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:19.641345 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c480851a-7f5c-453a-a938-fc298f5fa022 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:19.695650 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc07b92-3ba8-4a14-ac40-f9c80ba97801 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:19.697308 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc07b92-3ba8-4a14-ac40-f9c80ba97801 service neutron] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:19.707686 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 802/1621] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:12:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:12:19.712071 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc07b92-3ba8-4a14-ac40-f9c80ba97801 service neutron] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:19.720176 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c480851a-7f5c-453a-a938-fc298f5fa022 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:19.722277 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c480851a-7f5c-453a-a938-fc298f5fa022 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:19.746775 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 820/1622] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:12:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:12:19.746775 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c480851a-7f5c-453a-a938-fc298f5fa022 service neutron] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:19.752135 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c480851a-7f5c-453a-a938-fc298f5fa022 service neutron] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:19.757540 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9654ece5-46ee-452d-b7ba-aaafbc02075a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:19.758951 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9654ece5-46ee-452d-b7ba-aaafbc02075a admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:19.759308 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9654ece5-46ee-452d-b7ba-aaafbc02075a admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:19.759715 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9654ece5-46ee-452d-b7ba-aaafbc02075a admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:19.760433 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9654ece5-46ee-452d-b7ba-aaafbc02075a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:19.762969 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9654ece5-46ee-452d-b7ba-aaafbc02075a admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:19.817363 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc07b92-3ba8-4a14-ac40-f9c80ba97801 service neutron] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:19.823542 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c93bfd3-9f93-4655-8142-99a8b37549b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55723c756cae4e7ab636054db48de02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:19.824269 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c93bfd3-9f93-4655-8142-99a8b37549b4 service nova] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:19.824622 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c93bfd3-9f93-4655-8142-99a8b37549b4 service nova] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:19.824971 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c93bfd3-9f93-4655-8142-99a8b37549b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:19.841843 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:19.884379 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 821/1623] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:12:19 2022] POST /identity/v3/users => generated 297 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:12:19.884379 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c9398-3389-44ac-aba0-f16fb760c0c8 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:19.884379 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c9398-3389-44ac-aba0-f16fb760c0c8 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:19.884379 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c9398-3389-44ac-aba0-f16fb760c0c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:19.911630 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c93bfd3-9f93-4655-8142-99a8b37549b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:19.914698 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c93bfd3-9f93-4655-8142-99a8b37549b4 service nova] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:19.925204 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-bc2c9398-3389-44ac-aba0-f16fb760c0c8 None None] MFA Rules not processed for user `55723c756cae4e7ab636054db48de02d`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:19.926459 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-bc2c9398-3389-44ac-aba0-f16fb760c0c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:19.947680 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 803/1624] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:12:19 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:12:19.949127 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 822/1625] 10.222.0.82 () {58 vars in 1059 bytes} [Thu Aug 18 17:12:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:19.949611 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c93bfd3-9f93-4655-8142-99a8b37549b4 service nova] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:19.957074 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2c9398-3389-44ac-aba0-f16fb760c0c8 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:19.982484 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2c9398-3389-44ac-aba0-f16fb760c0c8 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:19.984248 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c93bfd3-9f93-4655-8142-99a8b37549b4 service nova] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:19.990214 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0144281e-c8e1-440d-ac6d-9e21ed42fcc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:19.994999 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0144281e-c8e1-440d-ac6d-9e21ed42fcc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:19.994999 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0144281e-c8e1-440d-ac6d-9e21ed42fcc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:19.994999 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0144281e-c8e1-440d-ac6d-9e21ed42fcc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:19.994999 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0144281e-c8e1-440d-ac6d-9e21ed42fcc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:19.995432 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09ecbeb-a5e3-43bd-88f3-2b1d8a8c3560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:19.995539 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0144281e-c8e1-440d-ac6d-9e21ed42fcc8 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:19.995626 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09ecbeb-a5e3-43bd-88f3-2b1d8a8c3560 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:19.995626 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09ecbeb-a5e3-43bd-88f3-2b1d8a8c3560 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:19.995626 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09ecbeb-a5e3-43bd-88f3-2b1d8a8c3560 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:19.995919 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09ecbeb-a5e3-43bd-88f3-2b1d8a8c3560 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:19.999728 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09ecbeb-a5e3-43bd-88f3-2b1d8a8c3560 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:20.010326 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 823/1626] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:12:19 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:12:20.010326 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0144281e-c8e1-440d-ac6d-9e21ed42fcc8 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:20.020583 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 804/1627] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:12:19 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:12:20.022050 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09ecbeb-a5e3-43bd-88f3-2b1d8a8c3560 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:20.035555 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0144281e-c8e1-440d-ac6d-9e21ed42fcc8 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:20.035945 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09ecbeb-a5e3-43bd-88f3-2b1d8a8c3560 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:20.042704 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1bb8b6-feff-4dca-99b0-8c52fd3e8803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:20.043678 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1bb8b6-feff-4dca-99b0-8c52fd3e8803 admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:20.044225 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1bb8b6-feff-4dca-99b0-8c52fd3e8803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:20.044368 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c796ea07-1609-40e6-92a0-c2abd8e922e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55723c756cae4e7ab636054db48de02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:20.045192 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c796ea07-1609-40e6-92a0-c2abd8e922e2 service nova] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:20.045573 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c796ea07-1609-40e6-92a0-c2abd8e922e2 service nova] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:20.045887 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1bb8b6-feff-4dca-99b0-8c52fd3e8803 admin admin] PATH_INFO: `/v3/projects/c4e093fc1c5b400fa82af0418419fb9c/users/339dda6cfd9d47df9d3686c64b1e4685/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:20.046048 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c796ea07-1609-40e6-92a0-c2abd8e922e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:20.064390 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1bb8b6-feff-4dca-99b0-8c52fd3e8803 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4e093fc1c5b400fa82af0418419fb9c, user_id=339dda6cfd9d47df9d3686c64b1e4685, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:20.067702 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1bb8b6-feff-4dca-99b0-8c52fd3e8803 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:20.080248 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:20.119715 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c796ea07-1609-40e6-92a0-c2abd8e922e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:20.121692 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c796ea07-1609-40e6-92a0-c2abd8e922e2 service nova] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:20.144715 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 824/1628] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:12:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:12:20.147609 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c796ea07-1609-40e6-92a0-c2abd8e922e2 service nova] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:20.148095 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 805/1629] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:12:20 2022] PUT /identity/v3/projects/c4e093fc1c5b400fa82af0418419fb9c/users/339dda6cfd9d47df9d3686c64b1e4685/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:12:20.149750 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1bb8b6-feff-4dca-99b0-8c52fd3e8803 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:20.171953 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c796ea07-1609-40e6-92a0-c2abd8e922e2 service nova] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:20.276538 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a36c58-62e8-4a32-b032-2c8cd1ffccc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:20.277348 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a36c58-62e8-4a32-b032-2c8cd1ffccc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:20.277852 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a36c58-62e8-4a32-b032-2c8cd1ffccc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:20.278342 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a36c58-62e8-4a32-b032-2c8cd1ffccc8 admin admin] PATH_INFO: `/v3/projects/3ad3ae716b75468cae4b02494944e0e2/users/0a066d988e4648e0847a8bcd406ddfe5/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:20.283719 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a36c58-62e8-4a32-b032-2c8cd1ffccc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ad3ae716b75468cae4b02494944e0e2, user_id=0a066d988e4648e0847a8bcd406ddfe5, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:20.287851 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a36c58-62e8-4a32-b032-2c8cd1ffccc8 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:20.289868 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1bb8b6-feff-4dca-99b0-8c52fd3e8803 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:20.294959 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4959479-89ab-4201-9dbf-78dadc9df01e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:20.295734 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4959479-89ab-4201-9dbf-78dadc9df01e admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:20.296559 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4959479-89ab-4201-9dbf-78dadc9df01e admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:20.296856 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4959479-89ab-4201-9dbf-78dadc9df01e admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:20.297654 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4959479-89ab-4201-9dbf-78dadc9df01e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:20.300094 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4959479-89ab-4201-9dbf-78dadc9df01e admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:20.303180 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 825/1630] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:12:20 2022] PUT /identity/v3/projects/3ad3ae716b75468cae4b02494944e0e2/users/0a066d988e4648e0847a8bcd406ddfe5/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:12:20.304724 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5ec3d-31de-4f02-b893-1529cd9e5be2 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:20.305351 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5ec3d-31de-4f02-b893-1529cd9e5be2 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:20.305725 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5ec3d-31de-4f02-b893-1529cd9e5be2 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:20.313392 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 826/1631] 10.222.0.82 () {54 vars in 912 bytes} [Thu Aug 18 17:12:20 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:12:20.317928 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c835b504-9af9-40f9-981c-9403284013a1 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:20.318214 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c835b504-9af9-40f9-981c-9403284013a1 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:20.318493 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c835b504-9af9-40f9-981c-9403284013a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:20.327465 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 806/1632] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:12:20 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:12:20.328898 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11423d45-f601-4a08-9bfd-9919e72ba497 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:20.329192 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11423d45-f601-4a08-9bfd-9919e72ba497 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:20.329471 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11423d45-f601-4a08-9bfd-9919e72ba497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:20.355899 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-11423d45-f601-4a08-9bfd-9919e72ba497 None None] MFA Rules not processed for user `55723c756cae4e7ab636054db48de02d`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:20.356983 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-11423d45-f601-4a08-9bfd-9919e72ba497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:20.371849 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-c835b504-9af9-40f9-981c-9403284013a1 None None] MFA Rules not processed for user `0a066d988e4648e0847a8bcd406ddfe5`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:20.374196 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-c835b504-9af9-40f9-981c-9403284013a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:20.467505 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 807/1633] 10.222.0.82 () {58 vars in 1009 bytes} [Thu Aug 18 17:12:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:20.474352 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11423d45-f601-4a08-9bfd-9919e72ba497 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:20.483291 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 827/1634] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:20 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:20.484914 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c835b504-9af9-40f9-981c-9403284013a1 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:20.493565 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:20.597086 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11423d45-f601-4a08-9bfd-9919e72ba497 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:20.608816 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:20.613389 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48bc843-807c-470c-9567-2a228f14ac1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:20.614774 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48bc843-807c-470c-9567-2a228f14ac1a admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:20.615434 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48bc843-807c-470c-9567-2a228f14ac1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:20.616369 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48bc843-807c-470c-9567-2a228f14ac1a admin admin] PATH_INFO: `/v3/projects/c4e093fc1c5b400fa82af0418419fb9c/users/339dda6cfd9d47df9d3686c64b1e4685/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:20.619071 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c835b504-9af9-40f9-981c-9403284013a1 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:20.633575 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48bc843-807c-470c-9567-2a228f14ac1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4e093fc1c5b400fa82af0418419fb9c, user_id=339dda6cfd9d47df9d3686c64b1e4685, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:20.634246 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cfa0f6-6c97-438d-b8b8-d3dd1fa7cdb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:20.634246 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cfa0f6-6c97-438d-b8b8-d3dd1fa7cdb5 service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:20.634246 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cfa0f6-6c97-438d-b8b8-d3dd1fa7cdb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:20.634246 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cfa0f6-6c97-438d-b8b8-d3dd1fa7cdb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:20.643346 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48bc843-807c-470c-9567-2a228f14ac1a admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:20.650253 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cfa0f6-6c97-438d-b8b8-d3dd1fa7cdb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:20.652070 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cfa0f6-6c97-438d-b8b8-d3dd1fa7cdb5 service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:20.665590 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 828/1635] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:12:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:12:20.666457 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cfa0f6-6c97-438d-b8b8-d3dd1fa7cdb5 service cinder] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:20.675896 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cfa0f6-6c97-438d-b8b8-d3dd1fa7cdb5 service cinder] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:20.683068 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 808/1636] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:12:20 2022] PUT /identity/v3/projects/c4e093fc1c5b400fa82af0418419fb9c/users/339dda6cfd9d47df9d3686c64b1e4685/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:12:20.684385 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48bc843-807c-470c-9567-2a228f14ac1a admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:20.741270 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693bd2b0-c38a-4e08-afb8-c0318f81ec56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:20.741871 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693bd2b0-c38a-4e08-afb8-c0318f81ec56 service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:20.742204 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693bd2b0-c38a-4e08-afb8-c0318f81ec56 service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:20.743050 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693bd2b0-c38a-4e08-afb8-c0318f81ec56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:20.801466 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48bc843-807c-470c-9567-2a228f14ac1a admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:20.810596 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366a07e7-804c-474f-8c92-be042a2a07dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55723c756cae4e7ab636054db48de02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:20.811550 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a07e7-804c-474f-8c92-be042a2a07dd service nova] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:20.811743 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a07e7-804c-474f-8c92-be042a2a07dd service nova] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:20.812093 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a07e7-804c-474f-8c92-be042a2a07dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:20.827442 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693bd2b0-c38a-4e08-afb8-c0318f81ec56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:20.830057 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693bd2b0-c38a-4e08-afb8-c0318f81ec56 service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:20.842827 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 829/1637] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:12:20 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:12:20.844466 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6a6b2c-609d-4cf8-bb95-215d528a49ae None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:20.844833 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6a6b2c-609d-4cf8-bb95-215d528a49ae None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:20.860414 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6a6b2c-609d-4cf8-bb95-215d528a49ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:20.928469 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-2c6a6b2c-609d-4cf8-bb95-215d528a49ae None None] MFA Rules not processed for user `339dda6cfd9d47df9d3686c64b1e4685`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:20.932045 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-2c6a6b2c-609d-4cf8-bb95-215d528a49ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:20.938421 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366a07e7-804c-474f-8c92-be042a2a07dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:20.941977 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366a07e7-804c-474f-8c92-be042a2a07dd service nova] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:20.967266 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 809/1638] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:12:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:12:21.016583 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e888f21-a6ad-4e57-b87a-f5b9f2b6ff93 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:21.017542 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e888f21-a6ad-4e57-b87a-f5b9f2b6ff93 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:21.017983 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e888f21-a6ad-4e57-b87a-f5b9f2b6ff93 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:21.021289 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 810/1639] 10.222.0.82 () {54 vars in 912 bytes} [Thu Aug 18 17:12:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:12:21.025256 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e130184-4879-4a57-98ca-0ef528ca862e None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:21.025747 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e130184-4879-4a57-98ca-0ef528ca862e None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:21.026196 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e130184-4879-4a57-98ca-0ef528ca862e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:21.035012 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 830/1640] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:20 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:21.097471 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-4e130184-4879-4a57-98ca-0ef528ca862e None None] MFA Rules not processed for user `55723c756cae4e7ab636054db48de02d`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:21.111182 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-4e130184-4879-4a57-98ca-0ef528ca862e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:21.114675 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 811/1641] 10.222.0.82 () {58 vars in 1009 bytes} [Thu Aug 18 17:12:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 17:12:21.180794 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6a6b2c-609d-4cf8-bb95-215d528a49ae None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:21.309891 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6a6b2c-609d-4cf8-bb95-215d528a49ae None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:21.309891 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34c172e-2c21-4c35-9507-c94736e10db5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:21.309891 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34c172e-2c21-4c35-9507-c94736e10db5 service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:21.309891 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34c172e-2c21-4c35-9507-c94736e10db5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:21.312888 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34c172e-2c21-4c35-9507-c94736e10db5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:21.312888 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34c172e-2c21-4c35-9507-c94736e10db5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:21.312888 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34c172e-2c21-4c35-9507-c94736e10db5 service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:21.312888 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 831/1642] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:12:21 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:12:21.349074 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e130184-4879-4a57-98ca-0ef528ca862e None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:21.364612 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e130184-4879-4a57-98ca-0ef528ca862e None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:21.371761 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd6a9af-689b-4458-bd13-bb380fd60cc6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:21.372402 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd6a9af-689b-4458-bd13-bb380fd60cc6 service cinder] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:21.373537 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd6a9af-689b-4458-bd13-bb380fd60cc6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:21.373874 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd6a9af-689b-4458-bd13-bb380fd60cc6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:21.389144 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd6a9af-689b-4458-bd13-bb380fd60cc6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:21.390826 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd6a9af-689b-4458-bd13-bb380fd60cc6 service cinder] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:21.411022 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 812/1643] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:12:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:12:21.958017 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34c172e-2c21-4c35-9507-c94736e10db5 service cinder] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:21.971101 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34c172e-2c21-4c35-9507-c94736e10db5 service cinder] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:21.976197 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3688f5-3ae7-41eb-9d34-b4dc42c148b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:21.977865 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3688f5-3ae7-41eb-9d34-b4dc42c148b5 service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:21.978248 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3688f5-3ae7-41eb-9d34-b4dc42c148b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:21.979989 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3688f5-3ae7-41eb-9d34-b4dc42c148b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:21.992626 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:22.068230 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3688f5-3ae7-41eb-9d34-b4dc42c148b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:22.069811 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3688f5-3ae7-41eb-9d34-b4dc42c148b5 service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:22.082319 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 832/1644] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:12:21 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:12:23.350833 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd6a9af-689b-4458-bd13-bb380fd60cc6 service cinder] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:23.378851 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:23.425230 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd6a9af-689b-4458-bd13-bb380fd60cc6 service cinder] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:23.432118 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d6b211-8b99-468c-8589-4179117207a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:23.433141 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d6b211-8b99-468c-8589-4179117207a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:23.433555 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d6b211-8b99-468c-8589-4179117207a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:23.433987 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d6b211-8b99-468c-8589-4179117207a6 admin admin] PATH_INFO: `/v3/users/29192b00b5c6486b942b49c5fb2641ce` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:23.435478 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d6b211-8b99-468c-8589-4179117207a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29192b00b5c6486b942b49c5fb2641ce)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:23.437807 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d6b211-8b99-468c-8589-4179117207a6 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:23.707134 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-03d6b211-8b99-468c-8589-4179117207a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29192b00b5c6486b942b49c5fb2641ce'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:23.722894 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-03d6b211-8b99-468c-8589-4179117207a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29192b00b5c6486b942b49c5fb2641ce'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:23.724460 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-03d6b211-8b99-468c-8589-4179117207a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29192b00b5c6486b942b49c5fb2641ce'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:23.823566 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-03d6b211-8b99-468c-8589-4179117207a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29192b00b5c6486b942b49c5fb2641ce'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:23.824204 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-03d6b211-8b99-468c-8589-4179117207a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29192b00b5c6486b942b49c5fb2641ce'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:23.839749 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 813/1645] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:12:23 2022] DELETE /identity/v3/users/29192b00b5c6486b942b49c5fb2641ce => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:12:23.843893 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bf0b25-12e0-4db6-aa98-d18694b59b45 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:23.844387 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bf0b25-12e0-4db6-aa98-d18694b59b45 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:23.844490 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bf0b25-12e0-4db6-aa98-d18694b59b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:23.911077 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-17bf0b25-12e0-4db6-aa98-d18694b59b45 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:23.916469 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-17bf0b25-12e0-4db6-aa98-d18694b59b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:24.042249 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d6b211-8b99-468c-8589-4179117207a6 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:24.043747 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-03d6b211-8b99-468c-8589-4179117207a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:24.120911 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 833/1646] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:24.135196 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bf0b25-12e0-4db6-aa98-d18694b59b45 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:24.137763 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-17bf0b25-12e0-4db6-aa98-d18694b59b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:24.221184 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d6b211-8b99-468c-8589-4179117207a6 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:24.232601 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5591a0-9e0e-41cd-88f7-ccb1843acb00 tempest-VolumeQuotaClassesTest-2059558801 tempest-VolumeQuotaClassesTest-2059558801-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '339dda6cfd9d47df9d3686c64b1e4685', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e093fc1c5b400fa82af0418419fb9c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:24.233237 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5591a0-9e0e-41cd-88f7-ccb1843acb00 tempest-VolumeQuotaClassesTest-2059558801 tempest-VolumeQuotaClassesTest-2059558801-project-admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:24.233529 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5591a0-9e0e-41cd-88f7-ccb1843acb00 tempest-VolumeQuotaClassesTest-2059558801 tempest-VolumeQuotaClassesTest-2059558801-project-admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:24.233810 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5591a0-9e0e-41cd-88f7-ccb1843acb00 tempest-VolumeQuotaClassesTest-2059558801 tempest-VolumeQuotaClassesTest-2059558801-project-admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:24.237581 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5591a0-9e0e-41cd-88f7-ccb1843acb00 tempest-VolumeQuotaClassesTest-2059558801 tempest-VolumeQuotaClassesTest-2059558801-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:12:24.237917 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5591a0-9e0e-41cd-88f7-ccb1843acb00 tempest-VolumeQuotaClassesTest-2059558801 tempest-VolumeQuotaClassesTest-2059558801-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:24.245193 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5591a0-9e0e-41cd-88f7-ccb1843acb00 tempest-VolumeQuotaClassesTest-2059558801 tempest-VolumeQuotaClassesTest-2059558801-project-admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:24.292352 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 814/1647] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:12:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 17:12:24.294458 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5591a0-9e0e-41cd-88f7-ccb1843acb00 tempest-VolumeQuotaClassesTest-2059558801 tempest-VolumeQuotaClassesTest-2059558801-project-admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:24.314532 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bf0b25-12e0-4db6-aa98-d18694b59b45 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:24.319172 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8278f0-464c-48e3-9a58-084671312e20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:24.319876 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8278f0-464c-48e3-9a58-084671312e20 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:24.320158 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8278f0-464c-48e3-9a58-084671312e20 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:24.321017 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8278f0-464c-48e3-9a58-084671312e20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:24.323795 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5591a0-9e0e-41cd-88f7-ccb1843acb00 tempest-VolumeQuotaClassesTest-2059558801 tempest-VolumeQuotaClassesTest-2059558801-project-admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:24.333241 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1284be0f-61b2-469b-aa94-08244fe3132d tempest-VolumeQuotaClassesTest-2059558801 tempest-VolumeQuotaClassesTest-2059558801-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '339dda6cfd9d47df9d3686c64b1e4685', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e093fc1c5b400fa82af0418419fb9c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:24.333875 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1284be0f-61b2-469b-aa94-08244fe3132d tempest-VolumeQuotaClassesTest-2059558801 tempest-VolumeQuotaClassesTest-2059558801-project-admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:24.334164 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1284be0f-61b2-469b-aa94-08244fe3132d tempest-VolumeQuotaClassesTest-2059558801 tempest-VolumeQuotaClassesTest-2059558801-project-admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:24.334443 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1284be0f-61b2-469b-aa94-08244fe3132d tempest-VolumeQuotaClassesTest-2059558801 tempest-VolumeQuotaClassesTest-2059558801-project-admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:24.334712 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8278f0-464c-48e3-9a58-084671312e20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:24.335115 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1284be0f-61b2-469b-aa94-08244fe3132d tempest-VolumeQuotaClassesTest-2059558801 tempest-VolumeQuotaClassesTest-2059558801-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:24.338405 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1284be0f-61b2-469b-aa94-08244fe3132d tempest-VolumeQuotaClassesTest-2059558801 tempest-VolumeQuotaClassesTest-2059558801-project-admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:24.338835 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8278f0-464c-48e3-9a58-084671312e20 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:24.358078 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 834/1648] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:12:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 17:12:24.442593 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 815/1649] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:12:24 2022] POST /identity/v3/projects => generated 355 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:12:24.447374 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8278f0-464c-48e3-9a58-084671312e20 service neutron] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:24.469055 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8278f0-464c-48e3-9a58-084671312e20 service neutron] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:24.752935 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:24.838533 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164385e3-6252-4dc0-bc21-da08c815331b tempest-VolumeQuotaClassesTest-2059558801 tempest-VolumeQuotaClassesTest-2059558801-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '339dda6cfd9d47df9d3686c64b1e4685', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e093fc1c5b400fa82af0418419fb9c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:24.839246 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164385e3-6252-4dc0-bc21-da08c815331b tempest-VolumeQuotaClassesTest-2059558801 tempest-VolumeQuotaClassesTest-2059558801-project-admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:24.839539 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164385e3-6252-4dc0-bc21-da08c815331b tempest-VolumeQuotaClassesTest-2059558801 tempest-VolumeQuotaClassesTest-2059558801-project-admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:24.839822 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164385e3-6252-4dc0-bc21-da08c815331b tempest-VolumeQuotaClassesTest-2059558801 tempest-VolumeQuotaClassesTest-2059558801-project-admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:24.841729 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164385e3-6252-4dc0-bc21-da08c815331b tempest-VolumeQuotaClassesTest-2059558801 tempest-VolumeQuotaClassesTest-2059558801-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:12:24.842092 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164385e3-6252-4dc0-bc21-da08c815331b tempest-VolumeQuotaClassesTest-2059558801 tempest-VolumeQuotaClassesTest-2059558801-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:24.843853 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164385e3-6252-4dc0-bc21-da08c815331b tempest-VolumeQuotaClassesTest-2059558801 tempest-VolumeQuotaClassesTest-2059558801-project-admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:24.890603 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 835/1650] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:12:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 444 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 17:12:25.495325 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1284be0f-61b2-469b-aa94-08244fe3132d tempest-VolumeQuotaClassesTest-2059558801 tempest-VolumeQuotaClassesTest-2059558801-project-admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:25.497766 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-1284be0f-61b2-469b-aa94-08244fe3132d tempest-VolumeQuotaClassesTest-2059558801 tempest-VolumeQuotaClassesTest-2059558801-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:25.522141 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:25.616689 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164385e3-6252-4dc0-bc21-da08c815331b tempest-VolumeQuotaClassesTest-2059558801 tempest-VolumeQuotaClassesTest-2059558801-project-admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:25.656974 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164385e3-6252-4dc0-bc21-da08c815331b tempest-VolumeQuotaClassesTest-2059558801 tempest-VolumeQuotaClassesTest-2059558801-project-admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:25.676664 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f72d0f-a29b-4f95-b90c-189d5f89d060 tempest-VolumeQuotaClassesTest-2059558801 tempest-VolumeQuotaClassesTest-2059558801-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '339dda6cfd9d47df9d3686c64b1e4685', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e093fc1c5b400fa82af0418419fb9c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:25.678451 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f72d0f-a29b-4f95-b90c-189d5f89d060 tempest-VolumeQuotaClassesTest-2059558801 tempest-VolumeQuotaClassesTest-2059558801-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:25.678746 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f72d0f-a29b-4f95-b90c-189d5f89d060 tempest-VolumeQuotaClassesTest-2059558801 tempest-VolumeQuotaClassesTest-2059558801-project-admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:25.679084 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f72d0f-a29b-4f95-b90c-189d5f89d060 tempest-VolumeQuotaClassesTest-2059558801 tempest-VolumeQuotaClassesTest-2059558801-project-admin] PATH_INFO: `/v3/projects/87ca05f2ce5545f981f7fd57fc7108b6` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:25.688922 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f72d0f-a29b-4f95-b90c-189d5f89d060 tempest-VolumeQuotaClassesTest-2059558801 tempest-VolumeQuotaClassesTest-2059558801-project-admin] RBAC: Authorizing `identity:delete_project(project_id=87ca05f2ce5545f981f7fd57fc7108b6)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:25.690473 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1284be0f-61b2-469b-aa94-08244fe3132d tempest-VolumeQuotaClassesTest-2059558801 tempest-VolumeQuotaClassesTest-2059558801-project-admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:25.694619 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f72d0f-a29b-4f95-b90c-189d5f89d060 tempest-VolumeQuotaClassesTest-2059558801 tempest-VolumeQuotaClassesTest-2059558801-project-admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:25.716884 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a8d857-20bb-4cd5-9bfe-3328824a5871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:25.719728 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8d857-20bb-4cd5-9bfe-3328824a5871 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:25.720376 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8d857-20bb-4cd5-9bfe-3328824a5871 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:25.720759 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8d857-20bb-4cd5-9bfe-3328824a5871 admin admin] PATH_INFO: `/v3/users/67673f940658480983950ba342dc77ae` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:25.722387 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a8d857-20bb-4cd5-9bfe-3328824a5871 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67673f940658480983950ba342dc77ae)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:25.725311 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a8d857-20bb-4cd5-9bfe-3328824a5871 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:25.968662 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-09a8d857-20bb-4cd5-9bfe-3328824a5871 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67673f940658480983950ba342dc77ae'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:25.972695 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-58f72d0f-a29b-4f95-b90c-189d5f89d060 tempest-VolumeQuotaClassesTest-2059558801 tempest-VolumeQuotaClassesTest-2059558801-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87ca05f2ce5545f981f7fd57fc7108b6', 'request_id': 'req-58f72d0f-a29b-4f95-b90c-189d5f89d060'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:25.989461 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-58f72d0f-a29b-4f95-b90c-189d5f89d060 tempest-VolumeQuotaClassesTest-2059558801 tempest-VolumeQuotaClassesTest-2059558801-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87ca05f2ce5545f981f7fd57fc7108b6', 'request_id': 'req-58f72d0f-a29b-4f95-b90c-189d5f89d060'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:25.991271 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-09a8d857-20bb-4cd5-9bfe-3328824a5871 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67673f940658480983950ba342dc77ae'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:25.995752 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-09a8d857-20bb-4cd5-9bfe-3328824a5871 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67673f940658480983950ba342dc77ae'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:26.126090 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-58f72d0f-a29b-4f95-b90c-189d5f89d060 tempest-VolumeQuotaClassesTest-2059558801 tempest-VolumeQuotaClassesTest-2059558801-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87ca05f2ce5545f981f7fd57fc7108b6', 'request_id': 'req-58f72d0f-a29b-4f95-b90c-189d5f89d060'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:26.144198 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-58f72d0f-a29b-4f95-b90c-189d5f89d060 tempest-VolumeQuotaClassesTest-2059558801 tempest-VolumeQuotaClassesTest-2059558801-project-admin] The token cache is being invalidate because project 87ca05f2ce5545f981f7fd57fc7108b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:12:26.144583 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-58f72d0f-a29b-4f95-b90c-189d5f89d060 tempest-VolumeQuotaClassesTest-2059558801 tempest-VolumeQuotaClassesTest-2059558801-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:26.145363 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-58f72d0f-a29b-4f95-b90c-189d5f89d060 tempest-VolumeQuotaClassesTest-2059558801 tempest-VolumeQuotaClassesTest-2059558801-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:26.150789 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 836/1651] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:12:25 2022] DELETE /identity/v3/projects/87ca05f2ce5545f981f7fd57fc7108b6 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:12:26.229916 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-09a8d857-20bb-4cd5-9bfe-3328824a5871 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67673f940658480983950ba342dc77ae'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:26.230624 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-09a8d857-20bb-4cd5-9bfe-3328824a5871 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67673f940658480983950ba342dc77ae'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:26.262966 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 816/1652] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:12:25 2022] DELETE /identity/v3/users/67673f940658480983950ba342dc77ae => generated 0 bytes in 768 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:12:26.269355 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ebda93-7a58-4b60-8187-4f5a938eb92c None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:26.269982 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ebda93-7a58-4b60-8187-4f5a938eb92c None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:26.270859 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ebda93-7a58-4b60-8187-4f5a938eb92c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:26.350170 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-95ebda93-7a58-4b60-8187-4f5a938eb92c None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:26.350170 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-95ebda93-7a58-4b60-8187-4f5a938eb92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:26.357861 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a8d857-20bb-4cd5-9bfe-3328824a5871 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:26.359628 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-09a8d857-20bb-4cd5-9bfe-3328824a5871 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:26.517058 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a8d857-20bb-4cd5-9bfe-3328824a5871 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:26.524896 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e887dd3c-f49b-4309-833b-e7e3178bb4bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:26.525978 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e887dd3c-f49b-4309-833b-e7e3178bb4bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:26.526467 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e887dd3c-f49b-4309-833b-e7e3178bb4bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:26.527006 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e887dd3c-f49b-4309-833b-e7e3178bb4bf admin admin] PATH_INFO: `/v3/projects/6299622d253a4d149d68b4bafdece4cf` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:26.528517 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e887dd3c-f49b-4309-833b-e7e3178bb4bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=6299622d253a4d149d68b4bafdece4cf)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:26.529898 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: WARNING py.warnings [None req-e887dd3c-f49b-4309-833b-e7e3178bb4bf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 18 17:12:26.529898 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: warnings.warn(msg) Aug 18 17:12:26.529898 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: Aug 18 17:12:26.531850 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e887dd3c-f49b-4309-833b-e7e3178bb4bf admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:26.603168 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 837/1653] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 17:12:26.628604 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ebda93-7a58-4b60-8187-4f5a938eb92c None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:26.633995 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-95ebda93-7a58-4b60-8187-4f5a938eb92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:26.732638 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-e887dd3c-f49b-4309-833b-e7e3178bb4bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6299622d253a4d149d68b4bafdece4cf', 'request_id': 'req-e887dd3c-f49b-4309-833b-e7e3178bb4bf'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:26.841889 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-e887dd3c-f49b-4309-833b-e7e3178bb4bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6299622d253a4d149d68b4bafdece4cf', 'request_id': 'req-e887dd3c-f49b-4309-833b-e7e3178bb4bf'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:26.852039 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-e887dd3c-f49b-4309-833b-e7e3178bb4bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6299622d253a4d149d68b4bafdece4cf', 'request_id': 'req-e887dd3c-f49b-4309-833b-e7e3178bb4bf'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:26.895641 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ebda93-7a58-4b60-8187-4f5a938eb92c None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:26.908580 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-e887dd3c-f49b-4309-833b-e7e3178bb4bf admin admin] The token cache is being invalidate because project 6299622d253a4d149d68b4bafdece4cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81650) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:12:26.908913 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-e887dd3c-f49b-4309-833b-e7e3178bb4bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:26.915817 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-e887dd3c-f49b-4309-833b-e7e3178bb4bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:26.924145 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 817/1654] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:12:26 2022] DELETE /identity/v3/projects/6299622d253a4d149d68b4bafdece4cf => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:12:26.929316 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cbd359-202e-4671-b9e7-50ba8238ca27 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:26.930156 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cbd359-202e-4671-b9e7-50ba8238ca27 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:26.932623 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cbd359-202e-4671-b9e7-50ba8238ca27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:26.995315 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:26.998028 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-26cbd359-202e-4671-b9e7-50ba8238ca27 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:26.999509 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-26cbd359-202e-4671-b9e7-50ba8238ca27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:27.057123 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dfad78-85a3-404d-b4ad-f25500f24a7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:27.057753 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dfad78-85a3-404d-b4ad-f25500f24a7b service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:27.058033 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dfad78-85a3-404d-b4ad-f25500f24a7b service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:27.058303 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dfad78-85a3-404d-b4ad-f25500f24a7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:27.062534 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-92dfad78-85a3-404d-b4ad-f25500f24a7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:27.145057 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 818/1655] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:27.149452 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d043678-a2c5-4ed0-8240-16fa68a27caf None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:27.149704 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d043678-a2c5-4ed0-8240-16fa68a27caf None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:27.149960 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d043678-a2c5-4ed0-8240-16fa68a27caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:27.203912 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-2d043678-a2c5-4ed0-8240-16fa68a27caf None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:27.205188 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-2d043678-a2c5-4ed0-8240-16fa68a27caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:27.217242 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 819/1656] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:27.221771 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d043678-a2c5-4ed0-8240-16fa68a27caf None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:27.227470 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dfad78-85a3-404d-b4ad-f25500f24a7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:27.232827 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dfad78-85a3-404d-b4ad-f25500f24a7b service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:27.258854 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d043678-a2c5-4ed0-8240-16fa68a27caf None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:27.265066 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 838/1657] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:12:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 637 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:12:27.271148 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dfad78-85a3-404d-b4ad-f25500f24a7b service neutron] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:27.275030 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-92dfad78-85a3-404d-b4ad-f25500f24a7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:27.279124 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62c84fc-19ae-44d2-88ab-f1890ee5bb03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:27.279984 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62c84fc-19ae-44d2-88ab-f1890ee5bb03 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:27.280484 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62c84fc-19ae-44d2-88ab-f1890ee5bb03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:27.280973 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62c84fc-19ae-44d2-88ab-f1890ee5bb03 admin admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:27.281769 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62c84fc-19ae-44d2-88ab-f1890ee5bb03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:12:27.286246 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62c84fc-19ae-44d2-88ab-f1890ee5bb03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:27.291198 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62c84fc-19ae-44d2-88ab-f1890ee5bb03 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:27.309849 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dfad78-85a3-404d-b4ad-f25500f24a7b service neutron] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:27.321229 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 820/1658] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:12:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 17:12:27.322429 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62c84fc-19ae-44d2-88ab-f1890ee5bb03 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:27.323698 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955f29b4-7854-49ea-a95a-929a30c4e663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:27.324480 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955f29b4-7854-49ea-a95a-929a30c4e663 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:27.324767 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955f29b4-7854-49ea-a95a-929a30c4e663 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:27.325043 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955f29b4-7854-49ea-a95a-929a30c4e663 admin admin] PATH_INFO: `/v3/users/3ebc6eeaa4a2479e9272298a4510b509` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:27.327823 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955f29b4-7854-49ea-a95a-929a30c4e663 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ebc6eeaa4a2479e9272298a4510b509)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:27.331421 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: WARNING py.warnings [None req-955f29b4-7854-49ea-a95a-929a30c4e663 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 18 17:12:27.331421 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: warnings.warn(msg) Aug 18 17:12:27.331421 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: Aug 18 17:12:27.334690 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955f29b4-7854-49ea-a95a-929a30c4e663 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:27.345395 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62c84fc-19ae-44d2-88ab-f1890ee5bb03 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:27.368665 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc42703f-9b98-44ca-a568-a2b3b12a6b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:27.369501 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc42703f-9b98-44ca-a568-a2b3b12a6b48 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:27.370372 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc42703f-9b98-44ca-a568-a2b3b12a6b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:27.370790 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc42703f-9b98-44ca-a568-a2b3b12a6b48 admin admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:27.371604 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc42703f-9b98-44ca-a568-a2b3b12a6b48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:27.374629 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc42703f-9b98-44ca-a568-a2b3b12a6b48 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:27.417878 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:27.509587 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 821/1659] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:12:27 2022] POST /identity/v3/projects => generated 377 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:12:27.514929 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc42703f-9b98-44ca-a568-a2b3b12a6b48 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:27.646571 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-955f29b4-7854-49ea-a95a-929a30c4e663 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ebc6eeaa4a2479e9272298a4510b509'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:27.665343 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc42703f-9b98-44ca-a568-a2b3b12a6b48 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:27.677030 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab34e86-4545-498a-851d-4befbf9f19b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:27.680656 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab34e86-4545-498a-851d-4befbf9f19b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:27.680656 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab34e86-4545-498a-851d-4befbf9f19b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:27.680656 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab34e86-4545-498a-851d-4befbf9f19b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:27.680656 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab34e86-4545-498a-851d-4befbf9f19b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:27.681474 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab34e86-4545-498a-851d-4befbf9f19b4 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:27.732015 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-955f29b4-7854-49ea-a95a-929a30c4e663 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ebc6eeaa4a2479e9272298a4510b509'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:27.732894 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-955f29b4-7854-49ea-a95a-929a30c4e663 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ebc6eeaa4a2479e9272298a4510b509'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:27.734071 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-955f29b4-7854-49ea-a95a-929a30c4e663 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ebc6eeaa4a2479e9272298a4510b509'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:27.754496 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-955f29b4-7854-49ea-a95a-929a30c4e663 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ebc6eeaa4a2479e9272298a4510b509'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:27.835980 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 822/1660] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:12:27 2022] POST /identity/v3/users => generated 289 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:12:27.837452 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab34e86-4545-498a-851d-4befbf9f19b4 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:27.841680 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-9ab34e86-4545-498a-851d-4befbf9f19b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:27.848865 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 839/1661] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:12:27 2022] DELETE /identity/v3/users/3ebc6eeaa4a2479e9272298a4510b509 => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:12:27.853772 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72acd60-7aad-45a9-80fa-13741dc8ecd9 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:27.854073 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72acd60-7aad-45a9-80fa-13741dc8ecd9 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:27.854352 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72acd60-7aad-45a9-80fa-13741dc8ecd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:27.877289 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab34e86-4545-498a-851d-4befbf9f19b4 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:27.906004 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-a72acd60-7aad-45a9-80fa-13741dc8ecd9 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:27.912242 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-a72acd60-7aad-45a9-80fa-13741dc8ecd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:28.062147 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 840/1662] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 17:12:28.092838 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6226858-51e9-440d-94a6-4338e0f81d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:28.093754 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6226858-51e9-440d-94a6-4338e0f81d94 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:28.094045 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6226858-51e9-440d-94a6-4338e0f81d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:28.094146 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72acd60-7aad-45a9-80fa-13741dc8ecd9 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:28.094325 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6226858-51e9-440d-94a6-4338e0f81d94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:28.095613 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-a72acd60-7aad-45a9-80fa-13741dc8ecd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:28.100980 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6226858-51e9-440d-94a6-4338e0f81d94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:28.102798 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6226858-51e9-440d-94a6-4338e0f81d94 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:28.141336 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 823/1663] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:12:27 2022] GET /identity/v3/roles => generated 1420 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 17:12:28.145215 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6226858-51e9-440d-94a6-4338e0f81d94 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:28.166152 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6226858-51e9-440d-94a6-4338e0f81d94 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:28.183209 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0245c340-891a-4269-8d56-864459bf0080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:28.183209 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0245c340-891a-4269-8d56-864459bf0080 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:28.183209 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0245c340-891a-4269-8d56-864459bf0080 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:28.183209 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0245c340-891a-4269-8d56-864459bf0080 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:28.186173 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0245c340-891a-4269-8d56-864459bf0080 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:28.193688 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0245c340-891a-4269-8d56-864459bf0080 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:28.230555 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 824/1664] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:12:28 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:12:28.234553 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0245c340-891a-4269-8d56-864459bf0080 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:28.261879 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0245c340-891a-4269-8d56-864459bf0080 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:28.277025 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0320dd2-eb58-4528-93e2-609fc04d7165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:28.277999 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0320dd2-eb58-4528-93e2-609fc04d7165 admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:28.278293 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0320dd2-eb58-4528-93e2-609fc04d7165 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:28.280686 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0320dd2-eb58-4528-93e2-609fc04d7165 admin admin] PATH_INFO: `/v3/projects/53ae2db4841d4c1da8b04af100612c64/users/1b4be4e285ad4556800f36754998b2ce/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:28.283991 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72acd60-7aad-45a9-80fa-13741dc8ecd9 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:28.296493 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5c43bf-b057-477b-9d60-65a17294653f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:28.296904 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c43bf-b057-477b-9d60-65a17294653f service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:28.297196 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c43bf-b057-477b-9d60-65a17294653f service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:28.297464 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c43bf-b057-477b-9d60-65a17294653f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:28.318298 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0320dd2-eb58-4528-93e2-609fc04d7165 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53ae2db4841d4c1da8b04af100612c64, user_id=1b4be4e285ad4556800f36754998b2ce, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:28.320067 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5c43bf-b057-477b-9d60-65a17294653f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:28.322254 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5c43bf-b057-477b-9d60-65a17294653f service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:28.326418 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0320dd2-eb58-4528-93e2-609fc04d7165 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:28.347367 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 841/1665] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:12:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 17:12:28.457804 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 825/1666] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:12:28 2022] PUT /identity/v3/projects/53ae2db4841d4c1da8b04af100612c64/users/1b4be4e285ad4556800f36754998b2ce/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:12:28.468810 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5969ef24-de1e-441a-8466-b8e7ef556e92 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:28.471253 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5969ef24-de1e-441a-8466-b8e7ef556e92 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:28.471772 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5969ef24-de1e-441a-8466-b8e7ef556e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:28.552914 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-5969ef24-de1e-441a-8466-b8e7ef556e92 None None] MFA Rules not processed for user `1b4be4e285ad4556800f36754998b2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:28.554189 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-5969ef24-de1e-441a-8466-b8e7ef556e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:28.681800 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 842/1667] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:28.686097 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0320dd2-eb58-4528-93e2-609fc04d7165 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:28.833732 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0320dd2-eb58-4528-93e2-609fc04d7165 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:28.843085 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb9b4a5-62e9-41d1-a2d6-9ffe986a48ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:28.844276 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9b4a5-62e9-41d1-a2d6-9ffe986a48ce admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:28.844786 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9b4a5-62e9-41d1-a2d6-9ffe986a48ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:28.845446 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9b4a5-62e9-41d1-a2d6-9ffe986a48ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:28.845944 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb9b4a5-62e9-41d1-a2d6-9ffe986a48ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:28.848324 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb9b4a5-62e9-41d1-a2d6-9ffe986a48ce admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:28.899336 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:28.974811 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5969ef24-de1e-441a-8466-b8e7ef556e92 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:28.976864 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-5969ef24-de1e-441a-8466-b8e7ef556e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:29.016315 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5969ef24-de1e-441a-8466-b8e7ef556e92 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:29.017380 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 826/1668] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:12:28 2022] POST /identity/v3/projects => generated 377 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:12:29.019053 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb9b4a5-62e9-41d1-a2d6-9ffe986a48ce admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:29.222507 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb9b4a5-62e9-41d1-a2d6-9ffe986a48ce admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:29.223792 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97367392-71d7-4fbf-832e-29832ec7cf7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:29.224342 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97367392-71d7-4fbf-832e-29832ec7cf7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:29.225315 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97367392-71d7-4fbf-832e-29832ec7cf7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:29.225315 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97367392-71d7-4fbf-832e-29832ec7cf7b admin admin] PATH_INFO: `/v3/projects/0534f4dae3d54429b998f82353578457` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:29.231791 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97367392-71d7-4fbf-832e-29832ec7cf7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0534f4dae3d54429b998f82353578457)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:29.233753 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97367392-71d7-4fbf-832e-29832ec7cf7b admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:29.261647 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658e78a7-fcb5-44aa-8576-cd121410ce77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:29.262285 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e78a7-fcb5-44aa-8576-cd121410ce77 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:29.262575 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e78a7-fcb5-44aa-8576-cd121410ce77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:29.262853 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e78a7-fcb5-44aa-8576-cd121410ce77 admin admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:29.263527 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658e78a7-fcb5-44aa-8576-cd121410ce77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:29.268159 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658e78a7-fcb5-44aa-8576-cd121410ce77 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:29.309331 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:29.432688 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 827/1669] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:12:29 2022] POST /identity/v3/users => generated 295 bytes in 415 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:12:29.435080 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658e78a7-fcb5-44aa-8576-cd121410ce77 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:29.459951 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658e78a7-fcb5-44aa-8576-cd121410ce77 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:29.474347 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b025d651-2c47-4ffa-924b-ec6de45c6586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:29.476775 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b025d651-2c47-4ffa-924b-ec6de45c6586 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:29.477820 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b025d651-2c47-4ffa-924b-ec6de45c6586 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:29.481280 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b025d651-2c47-4ffa-924b-ec6de45c6586 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:29.481280 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b025d651-2c47-4ffa-924b-ec6de45c6586 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:29.484049 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b025d651-2c47-4ffa-924b-ec6de45c6586 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:29.515070 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-97367392-71d7-4fbf-832e-29832ec7cf7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0534f4dae3d54429b998f82353578457', 'request_id': 'req-97367392-71d7-4fbf-832e-29832ec7cf7b'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:29.519694 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 828/1670] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:12:29 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:12:29.525130 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b025d651-2c47-4ffa-924b-ec6de45c6586 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:29.528942 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-97367392-71d7-4fbf-832e-29832ec7cf7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0534f4dae3d54429b998f82353578457', 'request_id': 'req-97367392-71d7-4fbf-832e-29832ec7cf7b'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:29.542707 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b025d651-2c47-4ffa-924b-ec6de45c6586 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:29.640029 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-97367392-71d7-4fbf-832e-29832ec7cf7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0534f4dae3d54429b998f82353578457', 'request_id': 'req-97367392-71d7-4fbf-832e-29832ec7cf7b'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:29.654847 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-97367392-71d7-4fbf-832e-29832ec7cf7b admin admin] The token cache is being invalidate because project 0534f4dae3d54429b998f82353578457 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:12:29.655180 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-97367392-71d7-4fbf-832e-29832ec7cf7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:29.664342 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-97367392-71d7-4fbf-832e-29832ec7cf7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:29.675560 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 843/1671] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:12:28 2022] DELETE /identity/v3/projects/0534f4dae3d54429b998f82353578457 => generated 0 bytes in 698 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:12:29.678269 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1070874-6b00-40a4-967c-dee188f22c89 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:29.678567 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1070874-6b00-40a4-967c-dee188f22c89 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:29.678845 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1070874-6b00-40a4-967c-dee188f22c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:29.692152 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55a2417-3315-45aa-a6c1-bc418a310021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:29.692779 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a2417-3315-45aa-a6c1-bc418a310021 admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:29.693067 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a2417-3315-45aa-a6c1-bc418a310021 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:29.693340 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a2417-3315-45aa-a6c1-bc418a310021 admin admin] PATH_INFO: `/v3/projects/1fd1c05e11f749558bab5ec8a0608220/users/951a1bc499fc4c2aa4f5d08fcf690ff5/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:29.778231 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55a2417-3315-45aa-a6c1-bc418a310021 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fd1c05e11f749558bab5ec8a0608220, user_id=951a1bc499fc4c2aa4f5d08fcf690ff5, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:29.782498 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-a1070874-6b00-40a4-967c-dee188f22c89 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:29.785092 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55a2417-3315-45aa-a6c1-bc418a310021 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:29.786570 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-a1070874-6b00-40a4-967c-dee188f22c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:29.823960 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 844/1672] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:29.832738 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8590da7e-371d-4343-a95f-712c12bea6cc None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:29.833040 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8590da7e-371d-4343-a95f-712c12bea6cc None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:29.833320 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8590da7e-371d-4343-a95f-712c12bea6cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:29.887004 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 829/1673] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:12:29 2022] PUT /identity/v3/projects/1fd1c05e11f749558bab5ec8a0608220/users/951a1bc499fc4c2aa4f5d08fcf690ff5/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:12:29.893053 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-8590da7e-371d-4343-a95f-712c12bea6cc None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:29.894451 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-8590da7e-371d-4343-a95f-712c12bea6cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:29.899063 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dbb829-a8f0-4a93-a1a6-52470d3779fb None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:29.899535 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dbb829-a8f0-4a93-a1a6-52470d3779fb None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:29.899818 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dbb829-a8f0-4a93-a1a6-52470d3779fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:29.989991 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-d2dbb829-a8f0-4a93-a1a6-52470d3779fb None None] MFA Rules not processed for user `22e737cba90b4d8890ecd692e6456470`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:29.991359 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-d2dbb829-a8f0-4a93-a1a6-52470d3779fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:29.995160 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 845/1674] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:29.996270 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8590da7e-371d-4343-a95f-712c12bea6cc None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:29.997851 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-8590da7e-371d-4343-a95f-712c12bea6cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:30.036051 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8590da7e-371d-4343-a95f-712c12bea6cc None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:30.043854 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2925f6-dda1-4f39-b654-7b0e36043c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:30.045791 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2925f6-dda1-4f39-b654-7b0e36043c8b admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:30.046276 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2925f6-dda1-4f39-b654-7b0e36043c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:30.046734 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2925f6-dda1-4f39-b654-7b0e36043c8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:30.047905 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2925f6-dda1-4f39-b654-7b0e36043c8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:30.049860 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2925f6-dda1-4f39-b654-7b0e36043c8b admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:30.074460 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 846/1675] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:12:29 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:12:30.075675 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2925f6-dda1-4f39-b654-7b0e36043c8b admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:30.103013 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2925f6-dda1-4f39-b654-7b0e36043c8b admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:30.108850 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa509da-5766-4eda-b060-0aec7f5c899c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:30.109808 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa509da-5766-4eda-b060-0aec7f5c899c admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:30.110347 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa509da-5766-4eda-b060-0aec7f5c899c admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:30.111128 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa509da-5766-4eda-b060-0aec7f5c899c admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:30.112037 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa509da-5766-4eda-b060-0aec7f5c899c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:12:30.112533 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa509da-5766-4eda-b060-0aec7f5c899c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:30.114598 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa509da-5766-4eda-b060-0aec7f5c899c admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:30.136631 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 847/1676] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:12:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 17:12:30.137903 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa509da-5766-4eda-b060-0aec7f5c899c admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:30.150640 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa509da-5766-4eda-b060-0aec7f5c899c admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:30.158473 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43680d55-8edf-45b6-8dfa-d877cc3a94b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:30.159345 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43680d55-8edf-45b6-8dfa-d877cc3a94b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:30.159668 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43680d55-8edf-45b6-8dfa-d877cc3a94b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:30.160779 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43680d55-8edf-45b6-8dfa-d877cc3a94b9 admin admin] PATH_INFO: `/v3/projects/1fd1c05e11f749558bab5ec8a0608220/users/951a1bc499fc4c2aa4f5d08fcf690ff5/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:30.160867 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 830/1677] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:29 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:30.165020 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dbb829-a8f0-4a93-a1a6-52470d3779fb None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:30.180073 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43680d55-8edf-45b6-8dfa-d877cc3a94b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fd1c05e11f749558bab5ec8a0608220, user_id=951a1bc499fc4c2aa4f5d08fcf690ff5, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:30.184843 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43680d55-8edf-45b6-8dfa-d877cc3a94b9 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:30.191593 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dbb829-a8f0-4a93-a1a6-52470d3779fb None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:30.199976 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 848/1678] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:12:30 2022] PUT /identity/v3/projects/1fd1c05e11f749558bab5ec8a0608220/users/951a1bc499fc4c2aa4f5d08fcf690ff5/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:12:30.201026 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43680d55-8edf-45b6-8dfa-d877cc3a94b9 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:30.204761 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-43680d55-8edf-45b6-8dfa-d877cc3a94b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:30.355235 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a61fdd-cbe8-4273-acc8-4835555f3ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:30.355881 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a61fdd-cbe8-4273-acc8-4835555f3ca4 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:30.357835 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a61fdd-cbe8-4273-acc8-4835555f3ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:30.358129 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a61fdd-cbe8-4273-acc8-4835555f3ca4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:30.360929 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a61fdd-cbe8-4273-acc8-4835555f3ca4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:30.367130 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a61fdd-cbe8-4273-acc8-4835555f3ca4 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:30.370627 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43680d55-8edf-45b6-8dfa-d877cc3a94b9 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:30.391790 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6116d5e-dbff-4b1d-8b18-13f14f18dcf7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55723c756cae4e7ab636054db48de02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:30.392680 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6116d5e-dbff-4b1d-8b18-13f14f18dcf7 service nova] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:30.393194 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6116d5e-dbff-4b1d-8b18-13f14f18dcf7 service nova] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:30.393669 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6116d5e-dbff-4b1d-8b18-13f14f18dcf7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:30.466743 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6116d5e-dbff-4b1d-8b18-13f14f18dcf7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:30.468745 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6116d5e-dbff-4b1d-8b18-13f14f18dcf7 service nova] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:30.493126 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 849/1679] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:12:30 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:12:30.494853 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00191929-b0cb-48e4-9b62-8cab68ba2b29 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:30.495475 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00191929-b0cb-48e4-9b62-8cab68ba2b29 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:30.496068 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00191929-b0cb-48e4-9b62-8cab68ba2b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:30.510094 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 831/1680] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:12:30 2022] POST /identity/v3/projects => generated 403 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:12:30.513055 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a61fdd-cbe8-4273-acc8-4835555f3ca4 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:30.533214 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-00191929-b0cb-48e4-9b62-8cab68ba2b29 None None] MFA Rules not processed for user `951a1bc499fc4c2aa4f5d08fcf690ff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:30.534955 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-00191929-b0cb-48e4-9b62-8cab68ba2b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:30.631640 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 850/1681] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:30 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:30.642293 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00191929-b0cb-48e4-9b62-8cab68ba2b29 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:30.644488 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-00191929-b0cb-48e4-9b62-8cab68ba2b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:30.690522 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a61fdd-cbe8-4273-acc8-4835555f3ca4 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:30.702415 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec8538b-78b4-41f6-9618-09ab7cdbbd21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:30.703301 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec8538b-78b4-41f6-9618-09ab7cdbbd21 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:30.703680 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec8538b-78b4-41f6-9618-09ab7cdbbd21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:30.704077 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec8538b-78b4-41f6-9618-09ab7cdbbd21 admin admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:30.704939 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec8538b-78b4-41f6-9618-09ab7cdbbd21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:30.711523 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec8538b-78b4-41f6-9618-09ab7cdbbd21 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:30.733775 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00191929-b0cb-48e4-9b62-8cab68ba2b29 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:30.740532 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c007fa5f-fdf5-4f12-84a2-4ca4784d3edb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:30.741691 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007fa5f-fdf5-4f12-84a2-4ca4784d3edb service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:30.742534 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007fa5f-fdf5-4f12-84a2-4ca4784d3edb service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:30.743134 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007fa5f-fdf5-4f12-84a2-4ca4784d3edb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:30.745693 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-c007fa5f-fdf5-4f12-84a2-4ca4784d3edb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:30.832439 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:30.834460 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 832/1682] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:12:30 2022] POST /identity/v3/users => generated 302 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:12:30.836493 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec8538b-78b4-41f6-9618-09ab7cdbbd21 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:30.853332 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-4ec8538b-78b4-41f6-9618-09ab7cdbbd21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:30.873251 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c007fa5f-fdf5-4f12-84a2-4ca4784d3edb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:30.877766 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c007fa5f-fdf5-4f12-84a2-4ca4784d3edb service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:30.886441 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec8538b-78b4-41f6-9618-09ab7cdbbd21 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:30.896268 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e7f8ac-85a3-4361-becd-9b2b2bf63991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:30.897342 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e7f8ac-85a3-4361-becd-9b2b2bf63991 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:30.897880 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e7f8ac-85a3-4361-becd-9b2b2bf63991 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:30.898994 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e7f8ac-85a3-4361-becd-9b2b2bf63991 admin admin] PATH_INFO: `/v3/projects/222d54f2729340c1beb7544f29288c3c` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:30.900492 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e7f8ac-85a3-4361-becd-9b2b2bf63991 admin admin] RBAC: Authorizing `identity:delete_project(project_id=222d54f2729340c1beb7544f29288c3c)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:30.901154 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 851/1683] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:12:30 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 17:12:30.903017 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c007fa5f-fdf5-4f12-84a2-4ca4784d3edb service cinder] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:30.917274 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c007fa5f-fdf5-4f12-84a2-4ca4784d3edb service cinder] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:30.919072 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e7f8ac-85a3-4361-becd-9b2b2bf63991 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:30.923612 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ef2eb3-c341-4671-b4c5-49bb69b682f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:30.924655 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef2eb3-c341-4671-b4c5-49bb69b682f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:30.925248 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef2eb3-c341-4671-b4c5-49bb69b682f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:30.925611 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef2eb3-c341-4671-b4c5-49bb69b682f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:30.926411 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ef2eb3-c341-4671-b4c5-49bb69b682f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:30.928419 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ef2eb3-c341-4671-b4c5-49bb69b682f4 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:30.965901 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 852/1684] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:12:30 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:12:30.979853 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ef2eb3-c341-4671-b4c5-49bb69b682f4 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:31.005456 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ef2eb3-c341-4671-b4c5-49bb69b682f4 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:31.019064 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70d1e8f-c499-444d-8597-36ab150f91a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:31.019817 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d1e8f-c499-444d-8597-36ab150f91a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:31.019980 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d1e8f-c499-444d-8597-36ab150f91a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:31.020282 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d1e8f-c499-444d-8597-36ab150f91a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:31.022117 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70d1e8f-c499-444d-8597-36ab150f91a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:31.024775 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70d1e8f-c499-444d-8597-36ab150f91a2 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:31.166358 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 853/1685] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:12:30 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:12:31.166358 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c27a6f-1bb7-4642-94dc-52ccc99fd53c None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:31.166358 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c27a6f-1bb7-4642-94dc-52ccc99fd53c None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:31.166358 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c27a6f-1bb7-4642-94dc-52ccc99fd53c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:31.224595 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-83c27a6f-1bb7-4642-94dc-52ccc99fd53c None None] MFA Rules not processed for user `e72104f038d24821be824a45e108d1ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:31.226206 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-83c27a6f-1bb7-4642-94dc-52ccc99fd53c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:31.298460 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 854/1686] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:31 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 17:12:31.299966 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c27a6f-1bb7-4642-94dc-52ccc99fd53c None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:31.315649 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c27a6f-1bb7-4642-94dc-52ccc99fd53c None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:31.328295 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-e2e7f8ac-85a3-4361-becd-9b2b2bf63991 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '222d54f2729340c1beb7544f29288c3c', 'request_id': 'req-e2e7f8ac-85a3-4361-becd-9b2b2bf63991'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:31.435246 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ec176f-340f-4b7d-8a66-60b743932d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:31.436899 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ec176f-340f-4b7d-8a66-60b743932d13 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:31.437441 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ec176f-340f-4b7d-8a66-60b743932d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:31.437976 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ec176f-340f-4b7d-8a66-60b743932d13 admin admin] PATH_INFO: `/v3/projects/318d93c0ba0f4501a5b7df9006720fc9/users/da378a9e2dfb4db58ad44d875b1da142/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:31.452185 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-e2e7f8ac-85a3-4361-becd-9b2b2bf63991 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '222d54f2729340c1beb7544f29288c3c', 'request_id': 'req-e2e7f8ac-85a3-4361-becd-9b2b2bf63991'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:31.462854 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-e2e7f8ac-85a3-4361-becd-9b2b2bf63991 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '222d54f2729340c1beb7544f29288c3c', 'request_id': 'req-e2e7f8ac-85a3-4361-becd-9b2b2bf63991'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:31.477214 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-e2e7f8ac-85a3-4361-becd-9b2b2bf63991 admin admin] The token cache is being invalidate because project 222d54f2729340c1beb7544f29288c3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81650) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:12:31.477437 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-e2e7f8ac-85a3-4361-becd-9b2b2bf63991 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:31.478156 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-e2e7f8ac-85a3-4361-becd-9b2b2bf63991 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:31.483059 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 833/1687] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:12:30 2022] DELETE /identity/v3/projects/222d54f2729340c1beb7544f29288c3c => generated 0 bytes in 645 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:12:31.483059 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b4e64-8b1e-4644-9190-03945e209658 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:31.483413 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b4e64-8b1e-4644-9190-03945e209658 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:31.483485 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b4e64-8b1e-4644-9190-03945e209658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:31.511319 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-d94b4e64-8b1e-4644-9190-03945e209658 None None] MFA Rules not processed for user `e72104f038d24821be824a45e108d1ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:31.517754 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ec176f-340f-4b7d-8a66-60b743932d13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=318d93c0ba0f4501a5b7df9006720fc9, user_id=da378a9e2dfb4db58ad44d875b1da142, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:31.520102 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-d94b4e64-8b1e-4644-9190-03945e209658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:31.526226 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ec176f-340f-4b7d-8a66-60b743932d13 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:31.635971 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 855/1688] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:12:31 2022] PUT /identity/v3/projects/318d93c0ba0f4501a5b7df9006720fc9/users/da378a9e2dfb4db58ad44d875b1da142/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:12:31.642150 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ec176f-340f-4b7d-8a66-60b743932d13 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:31.646181 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-23ec176f-340f-4b7d-8a66-60b743932d13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:31.706360 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 834/1689] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:31 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:31.710719 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a17d64-aade-42f4-999a-6e502e79e665 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:31.714084 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a17d64-aade-42f4-999a-6e502e79e665 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:31.714391 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a17d64-aade-42f4-999a-6e502e79e665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:31.731455 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:31.811338 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-f2a17d64-aade-42f4-999a-6e502e79e665 None None] MFA Rules not processed for user `da378a9e2dfb4db58ad44d875b1da142`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:31.812689 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-f2a17d64-aade-42f4-999a-6e502e79e665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:31.863853 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ec176f-340f-4b7d-8a66-60b743932d13 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:31.865656 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af7ac9e-b715-41df-9a8c-975b8142bab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:31.867367 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af7ac9e-b715-41df-9a8c-975b8142bab8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:31.867836 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af7ac9e-b715-41df-9a8c-975b8142bab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:31.868349 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af7ac9e-b715-41df-9a8c-975b8142bab8 admin admin] PATH_INFO: `/v3/users/339dda6cfd9d47df9d3686c64b1e4685` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:31.870461 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af7ac9e-b715-41df-9a8c-975b8142bab8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=339dda6cfd9d47df9d3686c64b1e4685)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:31.873106 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af7ac9e-b715-41df-9a8c-975b8142bab8 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:32.083354 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 835/1690] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:31 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:32.084841 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a17d64-aade-42f4-999a-6e502e79e665 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:32.088889 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-f2a17d64-aade-42f4-999a-6e502e79e665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:32.178005 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-7af7ac9e-b715-41df-9a8c-975b8142bab8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '339dda6cfd9d47df9d3686c64b1e4685'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:32.240163 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a17d64-aade-42f4-999a-6e502e79e665 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:32.258128 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a157f4c-0308-4a64-b8a7-20e8de1c257c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55723c756cae4e7ab636054db48de02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:32.258786 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a157f4c-0308-4a64-b8a7-20e8de1c257c service nova] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:32.259169 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a157f4c-0308-4a64-b8a7-20e8de1c257c service nova] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:32.259438 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a157f4c-0308-4a64-b8a7-20e8de1c257c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:32.262158 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-0a157f4c-0308-4a64-b8a7-20e8de1c257c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:32.295565 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-7af7ac9e-b715-41df-9a8c-975b8142bab8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '339dda6cfd9d47df9d3686c64b1e4685'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:32.296514 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-7af7ac9e-b715-41df-9a8c-975b8142bab8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '339dda6cfd9d47df9d3686c64b1e4685'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:32.297309 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-7af7ac9e-b715-41df-9a8c-975b8142bab8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '339dda6cfd9d47df9d3686c64b1e4685'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:32.312716 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-7af7ac9e-b715-41df-9a8c-975b8142bab8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '339dda6cfd9d47df9d3686c64b1e4685'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:32.386311 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 856/1691] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:12:31 2022] DELETE /identity/v3/users/339dda6cfd9d47df9d3686c64b1e4685 => generated 0 bytes in 745 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:12:32.386311 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af7ac9e-b715-41df-9a8c-975b8142bab8 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:32.390188 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-7af7ac9e-b715-41df-9a8c-975b8142bab8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:32.552584 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a157f4c-0308-4a64-b8a7-20e8de1c257c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:32.563843 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a157f4c-0308-4a64-b8a7-20e8de1c257c service nova] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:32.568475 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af7ac9e-b715-41df-9a8c-975b8142bab8 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:32.582769 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ee0c55-3be4-45ad-9599-283fb2a2d465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:32.584215 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ee0c55-3be4-45ad-9599-283fb2a2d465 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:32.584506 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ee0c55-3be4-45ad-9599-283fb2a2d465 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:32.584787 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ee0c55-3be4-45ad-9599-283fb2a2d465 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:32.586119 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ee0c55-3be4-45ad-9599-283fb2a2d465 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:32.588177 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ee0c55-3be4-45ad-9599-283fb2a2d465 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:32.594967 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 836/1692] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:12:32 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:12:32.620048 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:32.722774 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 857/1693] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:12:32 2022] POST /identity/v3/projects => generated 405 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:12:32.728011 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a157f4c-0308-4a64-b8a7-20e8de1c257c service nova] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:32.729759 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-0a157f4c-0308-4a64-b8a7-20e8de1c257c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:32.770312 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a157f4c-0308-4a64-b8a7-20e8de1c257c service nova] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:32.910670 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9591de2e-acf7-425c-a7ab-562285b86e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:32.911477 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591de2e-acf7-425c-a7ab-562285b86e28 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:32.911774 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591de2e-acf7-425c-a7ab-562285b86e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:32.912058 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591de2e-acf7-425c-a7ab-562285b86e28 admin admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:32.912737 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9591de2e-acf7-425c-a7ab-562285b86e28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:32.917548 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9591de2e-acf7-425c-a7ab-562285b86e28 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:32.926586 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:32.962812 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2042a-1182-4751-b4b7-f262332b5bc1 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:32.963526 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2042a-1182-4751-b4b7-f262332b5bc1 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:32.963892 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2042a-1182-4751-b4b7-f262332b5bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:33.006418 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-83a2042a-1182-4751-b4b7-f262332b5bc1 None None] MFA Rules not processed for user `ecdea0572cc64ca394cd4d977f536d5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:33.016119 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-83a2042a-1182-4751-b4b7-f262332b5bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:33.050829 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 837/1694] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:12:32 2022] POST /identity/v3/users => generated 309 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:12:33.054755 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9591de2e-acf7-425c-a7ab-562285b86e28 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:33.080145 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9591de2e-acf7-425c-a7ab-562285b86e28 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:33.096822 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256186f2-3fea-4b86-91e2-7a32bb8f448b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:33.097462 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256186f2-3fea-4b86-91e2-7a32bb8f448b admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:33.097748 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256186f2-3fea-4b86-91e2-7a32bb8f448b admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:33.098021 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256186f2-3fea-4b86-91e2-7a32bb8f448b admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:33.098672 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256186f2-3fea-4b86-91e2-7a32bb8f448b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:33.100347 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256186f2-3fea-4b86-91e2-7a32bb8f448b admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:33.128908 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 838/1695] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:12:33 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:12:33.132909 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256186f2-3fea-4b86-91e2-7a32bb8f448b admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:33.145051 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256186f2-3fea-4b86-91e2-7a32bb8f448b admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:33.150980 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e140d50-38e6-430e-bbdd-facdb70012b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:33.151674 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e140d50-38e6-430e-bbdd-facdb70012b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:33.151959 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e140d50-38e6-430e-bbdd-facdb70012b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:33.152237 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e140d50-38e6-430e-bbdd-facdb70012b4 admin admin] PATH_INFO: `/v3/projects/76c0629228a943bb8f513f775b351495/users/d0f28495791d42cf884897f318ff9abf/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:33.170056 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 858/1696] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:32 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:33.172903 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e140d50-38e6-430e-bbdd-facdb70012b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76c0629228a943bb8f513f775b351495, user_id=d0f28495791d42cf884897f318ff9abf, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:33.175511 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e140d50-38e6-430e-bbdd-facdb70012b4 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:33.186754 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f5d25d-749f-4872-bc28-459d701cf537 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:33.187133 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f5d25d-749f-4872-bc28-459d701cf537 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:33.187323 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f5d25d-749f-4872-bc28-459d701cf537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:33.216026 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-72f5d25d-749f-4872-bc28-459d701cf537 None None] MFA Rules not processed for user `ecdea0572cc64ca394cd4d977f536d5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:33.217381 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-72f5d25d-749f-4872-bc28-459d701cf537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:33.235862 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 859/1697] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:33 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:33.249811 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f5d25d-749f-4872-bc28-459d701cf537 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:33.253034 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-72f5d25d-749f-4872-bc28-459d701cf537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:33.276092 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 839/1698] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:12:33 2022] PUT /identity/v3/projects/76c0629228a943bb8f513f775b351495/users/d0f28495791d42cf884897f318ff9abf/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:12:33.278083 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e140d50-38e6-430e-bbdd-facdb70012b4 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:33.448648 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f5d25d-749f-4872-bc28-459d701cf537 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:33.464762 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e140d50-38e6-430e-bbdd-facdb70012b4 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:33.474723 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c036da-58bf-4a0d-b5d6-9bd505ff8dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:33.475566 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c036da-58bf-4a0d-b5d6-9bd505ff8dcf admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:33.477717 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c036da-58bf-4a0d-b5d6-9bd505ff8dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:33.479572 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c036da-58bf-4a0d-b5d6-9bd505ff8dcf admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:33.480367 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c036da-58bf-4a0d-b5d6-9bd505ff8dcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:33.488314 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c036da-58bf-4a0d-b5d6-9bd505ff8dcf admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:33.511289 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 840/1699] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:12:33 2022] GET /identity/v3/roles => generated 1420 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:12:33.511289 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c036da-58bf-4a0d-b5d6-9bd505ff8dcf admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:33.511289 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-66c036da-58bf-4a0d-b5d6-9bd505ff8dcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:33.552465 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c036da-58bf-4a0d-b5d6-9bd505ff8dcf admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:33.566055 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9a14f3-f61e-4ac5-adc7-76fa41aadba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:33.567230 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a14f3-f61e-4ac5-adc7-76fa41aadba3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:33.568455 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a14f3-f61e-4ac5-adc7-76fa41aadba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:33.568455 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a14f3-f61e-4ac5-adc7-76fa41aadba3 admin admin] PATH_INFO: `/v3/projects/c4e093fc1c5b400fa82af0418419fb9c` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:33.573101 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9a14f3-f61e-4ac5-adc7-76fa41aadba3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4e093fc1c5b400fa82af0418419fb9c)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:33.578266 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9a14f3-f61e-4ac5-adc7-76fa41aadba3 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:33.615414 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02a377a-33b6-41a8-93e5-ac2cf9a861a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55723c756cae4e7ab636054db48de02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:33.616565 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a377a-33b6-41a8-93e5-ac2cf9a861a8 service nova] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:33.616679 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a377a-33b6-41a8-93e5-ac2cf9a861a8 service nova] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:33.617848 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a377a-33b6-41a8-93e5-ac2cf9a861a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:33.703026 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02a377a-33b6-41a8-93e5-ac2cf9a861a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:33.704009 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02a377a-33b6-41a8-93e5-ac2cf9a861a8 service nova] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:33.715942 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 860/1700] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:12:33 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:12:33.716843 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02a377a-33b6-41a8-93e5-ac2cf9a861a8 service nova] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:33.773752 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02a377a-33b6-41a8-93e5-ac2cf9a861a8 service nova] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:33.785741 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-4c9a14f3-f61e-4ac5-adc7-76fa41aadba3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4e093fc1c5b400fa82af0418419fb9c', 'request_id': 'req-4c9a14f3-f61e-4ac5-adc7-76fa41aadba3'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:33.857828 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-4c9a14f3-f61e-4ac5-adc7-76fa41aadba3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4e093fc1c5b400fa82af0418419fb9c', 'request_id': 'req-4c9a14f3-f61e-4ac5-adc7-76fa41aadba3'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:33.864826 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-4c9a14f3-f61e-4ac5-adc7-76fa41aadba3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4e093fc1c5b400fa82af0418419fb9c', 'request_id': 'req-4c9a14f3-f61e-4ac5-adc7-76fa41aadba3'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:33.891755 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-4c9a14f3-f61e-4ac5-adc7-76fa41aadba3 admin admin] The token cache is being invalidate because project c4e093fc1c5b400fa82af0418419fb9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81650) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:12:33.895890 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-4c9a14f3-f61e-4ac5-adc7-76fa41aadba3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:33.895890 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-4c9a14f3-f61e-4ac5-adc7-76fa41aadba3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:33.903461 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 841/1701] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:12:33 2022] DELETE /identity/v3/projects/c4e093fc1c5b400fa82af0418419fb9c => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:12:33.909014 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5142f0-793a-463c-a5b6-b571cd2691d0 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:33.909301 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5142f0-793a-463c-a5b6-b571cd2691d0 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:33.909587 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5142f0-793a-463c-a5b6-b571cd2691d0 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:33.919866 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 842/1702] 10.222.0.82 () {54 vars in 912 bytes} [Thu Aug 18 17:12:33 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:12:33.929994 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f28fc8f-0ed3-443f-a3ce-4866b2399ba1 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:33.930447 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f28fc8f-0ed3-443f-a3ce-4866b2399ba1 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:33.930656 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f28fc8f-0ed3-443f-a3ce-4866b2399ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:33.944236 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e7b674-370b-41df-9284-53e4f476be85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:33.947095 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e7b674-370b-41df-9284-53e4f476be85 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:33.947384 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e7b674-370b-41df-9284-53e4f476be85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:33.947663 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e7b674-370b-41df-9284-53e4f476be85 admin admin] PATH_INFO: `/v3/projects/76c0629228a943bb8f513f775b351495/users/d0f28495791d42cf884897f318ff9abf/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:33.954541 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e7b674-370b-41df-9284-53e4f476be85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76c0629228a943bb8f513f775b351495, user_id=d0f28495791d42cf884897f318ff9abf, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:33.961017 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e7b674-370b-41df-9284-53e4f476be85 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:33.988801 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 861/1703] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:12:33 2022] PUT /identity/v3/projects/76c0629228a943bb8f513f775b351495/users/d0f28495791d42cf884897f318ff9abf/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:12:33.990323 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f3046-c159-4f94-a19a-55ddb849e6dd None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:33.995494 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f3046-c159-4f94-a19a-55ddb849e6dd None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:33.996108 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f3046-c159-4f94-a19a-55ddb849e6dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:34.018474 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-6f28fc8f-0ed3-443f-a3ce-4866b2399ba1 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:34.019654 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-6f28fc8f-0ed3-443f-a3ce-4866b2399ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:34.081849 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:34.090595 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-e96f3046-c159-4f94-a19a-55ddb849e6dd None None] MFA Rules not processed for user `fe2403ede53148b2a113ffa2a605e987`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:34.092042 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-e96f3046-c159-4f94-a19a-55ddb849e6dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:34.163551 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 843/1704] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:34.164435 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda13a3-ab9c-416c-8cd6-49400373a8e3 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:34.164735 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda13a3-ab9c-416c-8cd6-49400373a8e3 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:34.165000 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda13a3-ab9c-416c-8cd6-49400373a8e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:34.211578 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 862/1705] 10.222.0.82 () {58 vars in 1009 bytes} [Thu Aug 18 17:12:33 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:34.218165 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a377fb-9a03-4887-b9b0-3ab8656278e7 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:34.220439 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a377fb-9a03-4887-b9b0-3ab8656278e7 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:34.220439 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a377fb-9a03-4887-b9b0-3ab8656278e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:34.234359 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-3eda13a3-ab9c-416c-8cd6-49400373a8e3 None None] MFA Rules not processed for user `d0f28495791d42cf884897f318ff9abf`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:34.239594 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-3eda13a3-ab9c-416c-8cd6-49400373a8e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:34.260655 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-65a377fb-9a03-4887-b9b0-3ab8656278e7 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:34.262059 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-65a377fb-9a03-4887-b9b0-3ab8656278e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:34.296079 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 863/1706] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:34.296079 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a377fb-9a03-4887-b9b0-3ab8656278e7 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:34.297034 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-65a377fb-9a03-4887-b9b0-3ab8656278e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:34.328960 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a377fb-9a03-4887-b9b0-3ab8656278e7 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:34.351790 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13a3f5c-21c9-4f76-9fda-10a681daa220 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:34.351790 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13a3f5c-21c9-4f76-9fda-10a681daa220 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:34.351790 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13a3f5c-21c9-4f76-9fda-10a681daa220 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:34.351790 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13a3f5c-21c9-4f76-9fda-10a681daa220 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:34.357327 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13a3f5c-21c9-4f76-9fda-10a681daa220 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:34.359236 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13a3f5c-21c9-4f76-9fda-10a681daa220 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:34.377300 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 864/1707] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:12:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:12:34.378145 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13a3f5c-21c9-4f76-9fda-10a681daa220 service neutron] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:34.405816 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13a3f5c-21c9-4f76-9fda-10a681daa220 service neutron] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:34.412977 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195db050-ee7c-4625-baeb-461760e0cf35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:34.414809 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195db050-ee7c-4625-baeb-461760e0cf35 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:34.415481 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195db050-ee7c-4625-baeb-461760e0cf35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:34.415966 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195db050-ee7c-4625-baeb-461760e0cf35 admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:34.416927 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195db050-ee7c-4625-baeb-461760e0cf35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:12:34.417491 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195db050-ee7c-4625-baeb-461760e0cf35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:34.419351 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195db050-ee7c-4625-baeb-461760e0cf35 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:34.436932 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 865/1708] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:12:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 17:12:34.443011 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195db050-ee7c-4625-baeb-461760e0cf35 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:34.455442 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 844/1709] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:34 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:34.467579 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:34.477835 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195db050-ee7c-4625-baeb-461760e0cf35 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:34.493192 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b7cbd3-09e0-4f63-ac6f-f6cfbfad745b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:34.494124 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7cbd3-09e0-4f63-ac6f-f6cfbfad745b admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:34.494578 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7cbd3-09e0-4f63-ac6f-f6cfbfad745b admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:34.495189 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7cbd3-09e0-4f63-ac6f-f6cfbfad745b admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:34.496116 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b7cbd3-09e0-4f63-ac6f-f6cfbfad745b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:34.501283 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b7cbd3-09e0-4f63-ac6f-f6cfbfad745b admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:34.553418 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eda13a3-ab9c-416c-8cd6-49400373a8e3 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:34.555631 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-3eda13a3-ab9c-416c-8cd6-49400373a8e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:34.667985 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 866/1710] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:12:34 2022] POST /identity/v3/projects => generated 387 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:12:34.672235 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b7cbd3-09e0-4f63-ac6f-f6cfbfad745b admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:34.734462 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eda13a3-ab9c-416c-8cd6-49400373a8e3 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:34.819249 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b7cbd3-09e0-4f63-ac6f-f6cfbfad745b admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:34.873137 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc85d20-0132-48a3-aa8f-4239a7a03dd9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:34.873786 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc85d20-0132-48a3-aa8f-4239a7a03dd9 service cinder] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:34.874425 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc85d20-0132-48a3-aa8f-4239a7a03dd9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:34.874880 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc85d20-0132-48a3-aa8f-4239a7a03dd9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:34.878842 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea070eb-d577-44b0-ac2e-075841ec72b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:34.894947 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea070eb-d577-44b0-ac2e-075841ec72b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:34.896147 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea070eb-d577-44b0-ac2e-075841ec72b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:34.896445 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea070eb-d577-44b0-ac2e-075841ec72b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:34.898456 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea070eb-d577-44b0-ac2e-075841ec72b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:34.908539 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea070eb-d577-44b0-ac2e-075841ec72b9 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:34.949256 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:35.080474 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 867/1711] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:12:34 2022] POST /identity/v3/users => generated 294 bytes in 411 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 17:12:35.086026 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947cc1cd-cc93-47fb-9315-3b64a4c6d71e None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:35.086757 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947cc1cd-cc93-47fb-9315-3b64a4c6d71e None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:35.088982 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947cc1cd-cc93-47fb-9315-3b64a4c6d71e None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:35.090955 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc85d20-0132-48a3-aa8f-4239a7a03dd9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:35.094355 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc85d20-0132-48a3-aa8f-4239a7a03dd9 service cinder] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:35.098193 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 868/1712] 10.222.0.82 () {54 vars in 912 bytes} [Thu Aug 18 17:12:35 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:12:35.103808 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947cc1cd-cc93-47fb-9315-3b64a4c6d71e None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:35.151641 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 845/1713] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:12:34 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 599 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 17:12:35.160834 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97af84c-3642-4fa3-9e29-b727f5cc2771 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:35.162000 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97af84c-3642-4fa3-9e29-b727f5cc2771 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:35.164626 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947cc1cd-cc93-47fb-9315-3b64a4c6d71e None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:35.164956 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97af84c-3642-4fa3-9e29-b727f5cc2771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:35.192511 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb074b-151a-4daf-a6c7-4efd0743cd75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:35.195773 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb074b-151a-4daf-a6c7-4efd0743cd75 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:35.196066 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb074b-151a-4daf-a6c7-4efd0743cd75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:35.196312 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb074b-151a-4daf-a6c7-4efd0743cd75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:35.197769 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcb074b-151a-4daf-a6c7-4efd0743cd75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:35.204657 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcb074b-151a-4daf-a6c7-4efd0743cd75 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:35.239236 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 869/1714] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:12:35 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:12:35.242403 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-d97af84c-3642-4fa3-9e29-b727f5cc2771 None None] MFA Rules not processed for user `fe2403ede53148b2a113ffa2a605e987`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:35.242641 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb074b-151a-4daf-a6c7-4efd0743cd75 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:35.244535 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-d97af84c-3642-4fa3-9e29-b727f5cc2771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:35.264265 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb074b-151a-4daf-a6c7-4efd0743cd75 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:35.282639 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc7872c-1206-43b8-978b-a065704e79f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:35.283305 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7872c-1206-43b8-978b-a065704e79f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:35.283590 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7872c-1206-43b8-978b-a065704e79f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:35.283871 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7872c-1206-43b8-978b-a065704e79f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:35.284520 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc7872c-1206-43b8-978b-a065704e79f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:35.286165 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc7872c-1206-43b8-978b-a065704e79f4 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:35.327903 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 870/1715] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:12:35 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:12:35.335498 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc7872c-1206-43b8-978b-a065704e79f4 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:35.374624 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc7872c-1206-43b8-978b-a065704e79f4 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:35.394616 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc171ee4-9199-4995-b388-972686918cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:35.395338 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc171ee4-9199-4995-b388-972686918cff admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:35.395620 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc171ee4-9199-4995-b388-972686918cff admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:35.395908 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc171ee4-9199-4995-b388-972686918cff admin admin] PATH_INFO: `/v3/projects/264d1e5aa00a4d5e9b51a39a5315737c/users/149667be33e84383b5a6d96c74781bed/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:35.408901 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 846/1716] 10.222.0.82 () {58 vars in 1009 bytes} [Thu Aug 18 17:12:35 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:35.421439 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97af84c-3642-4fa3-9e29-b727f5cc2771 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:35.432231 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97af84c-3642-4fa3-9e29-b727f5cc2771 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:35.436341 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfff992a-c886-4f5b-9d72-4ea3b79e086a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:35.437794 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff992a-c886-4f5b-9d72-4ea3b79e086a service neutron] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:35.437794 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff992a-c886-4f5b-9d72-4ea3b79e086a service neutron] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:35.437794 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff992a-c886-4f5b-9d72-4ea3b79e086a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:35.441211 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc171ee4-9199-4995-b388-972686918cff admin admin] RBAC: Authorizing `identity:create_grant(project_id=264d1e5aa00a4d5e9b51a39a5315737c, user_id=149667be33e84383b5a6d96c74781bed, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:35.443973 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc171ee4-9199-4995-b388-972686918cff admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:35.447138 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfff992a-c886-4f5b-9d72-4ea3b79e086a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:35.448927 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfff992a-c886-4f5b-9d72-4ea3b79e086a service neutron] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:35.459659 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 847/1717] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:12:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 17:12:35.539672 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 871/1718] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:12:35 2022] PUT /identity/v3/projects/264d1e5aa00a4d5e9b51a39a5315737c/users/149667be33e84383b5a6d96c74781bed/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:12:35.554763 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f096afe-1715-4dba-aec0-376a9c24e0df None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:35.555239 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f096afe-1715-4dba-aec0-376a9c24e0df None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:35.555590 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f096afe-1715-4dba-aec0-376a9c24e0df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:35.640013 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-9f096afe-1715-4dba-aec0-376a9c24e0df None None] MFA Rules not processed for user `149667be33e84383b5a6d96c74781bed`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:35.643850 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-9f096afe-1715-4dba-aec0-376a9c24e0df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:35.708324 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647faa32-4f0a-49e8-b891-4e8cb6f3f371 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:35.709548 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647faa32-4f0a-49e8-b891-4e8cb6f3f371 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:35.710044 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647faa32-4f0a-49e8-b891-4e8cb6f3f371 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:35.712152 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 872/1719] 10.222.0.82 () {54 vars in 961 bytes} [Thu Aug 18 17:12:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:12:35.726202 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234b098e-4710-4414-ab55-2c55a0616d35 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:35.726202 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234b098e-4710-4414-ab55-2c55a0616d35 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:35.726202 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234b098e-4710-4414-ab55-2c55a0616d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:35.781917 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-234b098e-4710-4414-ab55-2c55a0616d35 None None] MFA Rules not processed for user `29653ba1a2a74232972ef7a9000518b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:35.785856 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-234b098e-4710-4414-ab55-2c55a0616d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:35.893364 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:35.893920 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 848/1720] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:35 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:35.896278 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f096afe-1715-4dba-aec0-376a9c24e0df None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:36.025499 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 873/1721] 10.222.0.82 () {58 vars in 1058 bytes} [Thu Aug 18 17:12:35 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Aug 18 17:12:36.033291 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234b098e-4710-4414-ab55-2c55a0616d35 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:36.051062 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234b098e-4710-4414-ab55-2c55a0616d35 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:36.059176 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a70f2ef-fb0d-44c0-9f6f-d8edc24c6205 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29653ba1a2a74232972ef7a9000518b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:36.059905 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a70f2ef-fb0d-44c0-9f6f-d8edc24c6205 service placement] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:36.060499 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a70f2ef-fb0d-44c0-9f6f-d8edc24c6205 service placement] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:36.060620 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a70f2ef-fb0d-44c0-9f6f-d8edc24c6205 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:36.063745 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-1a70f2ef-fb0d-44c0-9f6f-d8edc24c6205 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:36.081420 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a70f2ef-fb0d-44c0-9f6f-d8edc24c6205 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:36.084008 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a70f2ef-fb0d-44c0-9f6f-d8edc24c6205 service placement] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:36.096521 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f096afe-1715-4dba-aec0-376a9c24e0df None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:36.106882 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42bad69-0cb7-41ad-9e11-b3b429889d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:36.106882 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42bad69-0cb7-41ad-9e11-b3b429889d68 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:36.106882 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42bad69-0cb7-41ad-9e11-b3b429889d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:36.106882 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42bad69-0cb7-41ad-9e11-b3b429889d68 admin admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:36.107181 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42bad69-0cb7-41ad-9e11-b3b429889d68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:36.107301 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42bad69-0cb7-41ad-9e11-b3b429889d68 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:36.131261 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 874/1722] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:12:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 17:12:36.140995 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:36.256407 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a70f2ef-fb0d-44c0-9f6f-d8edc24c6205 service placement] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:36.258549 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 849/1723] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:12:35 2022] POST /identity/v3/projects => generated 389 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:12:36.276624 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a70f2ef-fb0d-44c0-9f6f-d8edc24c6205 service placement] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:36.337700 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:36.439201 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ae6971-64e2-4ba7-8bdd-59a3affaf810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:36.444738 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae6971-64e2-4ba7-8bdd-59a3affaf810 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:36.444738 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae6971-64e2-4ba7-8bdd-59a3affaf810 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:36.444738 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae6971-64e2-4ba7-8bdd-59a3affaf810 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:36.444738 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ae6971-64e2-4ba7-8bdd-59a3affaf810 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:36.445524 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ae6971-64e2-4ba7-8bdd-59a3affaf810 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:36.505337 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc149e7-4896-4dc6-a8ce-e020e65c48d0 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:36.505695 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc149e7-4896-4dc6-a8ce-e020e65c48d0 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:36.506065 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc149e7-4896-4dc6-a8ce-e020e65c48d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:36.542805 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-0dc149e7-4896-4dc6-a8ce-e020e65c48d0 None None] MFA Rules not processed for user `22e737cba90b4d8890ecd692e6456470`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:36.548277 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-0dc149e7-4896-4dc6-a8ce-e020e65c48d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:36.576746 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 875/1724] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:12:36 2022] POST /identity/v3/users => generated 301 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 17:12:36.580859 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ae6971-64e2-4ba7-8bdd-59a3affaf810 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:36.594387 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ae6971-64e2-4ba7-8bdd-59a3affaf810 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:36.601822 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0657e37a-a3a4-43e0-9459-e7503f0c90a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:36.605550 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0657e37a-a3a4-43e0-9459-e7503f0c90a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:36.605550 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0657e37a-a3a4-43e0-9459-e7503f0c90a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:36.605550 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0657e37a-a3a4-43e0-9459-e7503f0c90a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:36.605550 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0657e37a-a3a4-43e0-9459-e7503f0c90a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:36.615957 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0657e37a-a3a4-43e0-9459-e7503f0c90a9 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:36.635686 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 850/1725] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:36 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:36.662425 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 876/1726] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:12:36 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:12:36.668623 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc149e7-4896-4dc6-a8ce-e020e65c48d0 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:36.688764 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc149e7-4896-4dc6-a8ce-e020e65c48d0 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:36.701169 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92ed7d1-4a0c-4681-90e4-2eee66209ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:36.701943 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ed7d1-4a0c-4681-90e4-2eee66209ba3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:36.702294 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ed7d1-4a0c-4681-90e4-2eee66209ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:36.702636 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ed7d1-4a0c-4681-90e4-2eee66209ba3 admin admin] PATH_INFO: `/v3/projects/9e0160d0266041cfab13e00bf8253412/users/c585bef1854f4737a82bf39796263a61/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:36.761566 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92ed7d1-4a0c-4681-90e4-2eee66209ba3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e0160d0266041cfab13e00bf8253412, user_id=c585bef1854f4737a82bf39796263a61, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:36.765385 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92ed7d1-4a0c-4681-90e4-2eee66209ba3 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:36.950773 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b8cfd4-a95f-4f77-92d1-c02d3e108579 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:36.950773 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b8cfd4-a95f-4f77-92d1-c02d3e108579 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:36.950773 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b8cfd4-a95f-4f77-92d1-c02d3e108579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:36.991150 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-51b8cfd4-a95f-4f77-92d1-c02d3e108579 None None] MFA Rules not processed for user `ecdea0572cc64ca394cd4d977f536d5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:36.993216 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-51b8cfd4-a95f-4f77-92d1-c02d3e108579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:37.040109 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 851/1727] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:12:36 2022] PUT /identity/v3/projects/9e0160d0266041cfab13e00bf8253412/users/c585bef1854f4737a82bf39796263a61/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:12:37.044221 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92ed7d1-4a0c-4681-90e4-2eee66209ba3 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:37.172874 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 877/1728] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:36 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:37.209743 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92ed7d1-4a0c-4681-90e4-2eee66209ba3 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:37.235715 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e932abad-b7c6-4441-94a0-e411ac178882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:37.236873 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e932abad-b7c6-4441-94a0-e411ac178882 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:37.237402 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e932abad-b7c6-4441-94a0-e411ac178882 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:37.237866 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e932abad-b7c6-4441-94a0-e411ac178882 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:37.238757 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e932abad-b7c6-4441-94a0-e411ac178882 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:37.240817 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e932abad-b7c6-4441-94a0-e411ac178882 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:37.257452 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 852/1729] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:12:37 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 17:12:37.262739 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b8cfd4-a95f-4f77-92d1-c02d3e108579 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:37.287148 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b8cfd4-a95f-4f77-92d1-c02d3e108579 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:37.288633 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbab1633-74ff-4e75-a870-00fb53eeccc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:37.289403 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbab1633-74ff-4e75-a870-00fb53eeccc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:37.289774 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbab1633-74ff-4e75-a870-00fb53eeccc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:37.290110 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbab1633-74ff-4e75-a870-00fb53eeccc7 admin admin] PATH_INFO: `/v3/projects/9e0160d0266041cfab13e00bf8253412/users/c585bef1854f4737a82bf39796263a61/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:37.292562 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbab1633-74ff-4e75-a870-00fb53eeccc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e0160d0266041cfab13e00bf8253412, user_id=c585bef1854f4737a82bf39796263a61, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:37.295565 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbab1633-74ff-4e75-a870-00fb53eeccc7 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:37.318463 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 878/1730] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:12:37 2022] PUT /identity/v3/projects/9e0160d0266041cfab13e00bf8253412/users/c585bef1854f4737a82bf39796263a61/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:12:37.321195 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbf8586-4002-4df3-b99b-a000ac8529c4 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:37.321499 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbf8586-4002-4df3-b99b-a000ac8529c4 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:37.321779 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbf8586-4002-4df3-b99b-a000ac8529c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:37.343198 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbab1633-74ff-4e75-a870-00fb53eeccc7 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:37.350365 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-bbab1633-74ff-4e75-a870-00fb53eeccc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:37.399498 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-fcbf8586-4002-4df3-b99b-a000ac8529c4 None None] MFA Rules not processed for user `c585bef1854f4737a82bf39796263a61`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:37.401650 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-fcbf8586-4002-4df3-b99b-a000ac8529c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:37.483416 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbab1633-74ff-4e75-a870-00fb53eeccc7 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:37.495597 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da4afbc-1eff-4118-aea6-9e4c77677ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:37.496908 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da4afbc-1eff-4118-aea6-9e4c77677ee7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:37.497247 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da4afbc-1eff-4118-aea6-9e4c77677ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:37.497532 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da4afbc-1eff-4118-aea6-9e4c77677ee7 admin admin] PATH_INFO: `/v3/users/da378a9e2dfb4db58ad44d875b1da142` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:37.499471 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da4afbc-1eff-4118-aea6-9e4c77677ee7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da378a9e2dfb4db58ad44d875b1da142)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:37.501810 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da4afbc-1eff-4118-aea6-9e4c77677ee7 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:37.509747 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:37.563631 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 853/1731] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:37 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:37.576188 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbf8586-4002-4df3-b99b-a000ac8529c4 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:37.578128 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-fcbf8586-4002-4df3-b99b-a000ac8529c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:37.661253 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbf8586-4002-4df3-b99b-a000ac8529c4 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:37.666656 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb6929e-9c72-444a-9d8c-cf4fab1a64a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:37.681402 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6929e-9c72-444a-9d8c-cf4fab1a64a5 service cinder] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:37.681402 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6929e-9c72-444a-9d8c-cf4fab1a64a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:37.681613 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6929e-9c72-444a-9d8c-cf4fab1a64a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:37.714374 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb6929e-9c72-444a-9d8c-cf4fab1a64a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:37.714374 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb6929e-9c72-444a-9d8c-cf4fab1a64a5 service cinder] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:37.723216 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 854/1732] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:12:37 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 17:12:37.829588 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:37.853507 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-6da4afbc-1eff-4118-aea6-9e4c77677ee7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da378a9e2dfb4db58ad44d875b1da142'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:37.959226 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb6929e-9c72-444a-9d8c-cf4fab1a64a5 service cinder] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:37.974018 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb6929e-9c72-444a-9d8c-cf4fab1a64a5 service cinder] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:37.979951 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23aabcd-96f8-453b-b9af-155ff06c67b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:37.981837 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23aabcd-96f8-453b-b9af-155ff06c67b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:37.982376 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23aabcd-96f8-453b-b9af-155ff06c67b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:37.982834 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23aabcd-96f8-453b-b9af-155ff06c67b2 admin admin] PATH_INFO: `/v3/users/149667be33e84383b5a6d96c74781bed` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:37.984696 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23aabcd-96f8-453b-b9af-155ff06c67b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=149667be33e84383b5a6d96c74781bed)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:37.995640 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-6da4afbc-1eff-4118-aea6-9e4c77677ee7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da378a9e2dfb4db58ad44d875b1da142'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:37.996510 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23aabcd-96f8-453b-b9af-155ff06c67b2 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:37.997508 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-6da4afbc-1eff-4118-aea6-9e4c77677ee7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da378a9e2dfb4db58ad44d875b1da142'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:37.999189 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-6da4afbc-1eff-4118-aea6-9e4c77677ee7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da378a9e2dfb4db58ad44d875b1da142'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:38.017466 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-6da4afbc-1eff-4118-aea6-9e4c77677ee7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da378a9e2dfb4db58ad44d875b1da142'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:38.105935 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 879/1733] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:12:37 2022] DELETE /identity/v3/users/da378a9e2dfb4db58ad44d875b1da142 => generated 0 bytes in 763 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:12:38.110401 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b268e-9865-4892-acc3-fdf016680e78 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:38.114976 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b268e-9865-4892-acc3-fdf016680e78 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:38.115560 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b268e-9865-4892-acc3-fdf016680e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:38.146022 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-294b268e-9865-4892-acc3-fdf016680e78 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:38.148268 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-294b268e-9865-4892-acc3-fdf016680e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:38.217681 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:38.298438 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 880/1734] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:38.309450 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-e23aabcd-96f8-453b-b9af-155ff06c67b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '149667be33e84383b5a6d96c74781bed'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:38.312075 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294b268e-9865-4892-acc3-fdf016680e78 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:38.313826 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-294b268e-9865-4892-acc3-fdf016680e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:38.352660 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-e23aabcd-96f8-453b-b9af-155ff06c67b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '149667be33e84383b5a6d96c74781bed'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:38.353513 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-e23aabcd-96f8-453b-b9af-155ff06c67b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '149667be33e84383b5a6d96c74781bed'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:38.437301 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294b268e-9865-4892-acc3-fdf016680e78 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:38.452141 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c33db9-f752-4cf3-992c-caab2b3b7860 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:38.452141 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c33db9-f752-4cf3-992c-caab2b3b7860 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:38.452141 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c33db9-f752-4cf3-992c-caab2b3b7860 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:38.452141 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c33db9-f752-4cf3-992c-caab2b3b7860 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:38.466335 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c33db9-f752-4cf3-992c-caab2b3b7860 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:38.467910 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c33db9-f752-4cf3-992c-caab2b3b7860 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:38.476789 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-e23aabcd-96f8-453b-b9af-155ff06c67b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '149667be33e84383b5a6d96c74781bed'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:38.477436 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-e23aabcd-96f8-453b-b9af-155ff06c67b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '149667be33e84383b5a6d96c74781bed'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:38.480851 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 881/1735] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:12:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:12:38.503574 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:38.527334 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 855/1736] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:12:37 2022] DELETE /identity/v3/users/149667be33e84383b5a6d96c74781bed => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:12:38.531574 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d201441-31ea-4ebd-bfcd-281c4ed16fc9 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:38.532233 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d201441-31ea-4ebd-bfcd-281c4ed16fc9 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:38.532978 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d201441-31ea-4ebd-bfcd-281c4ed16fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:38.558288 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-0d201441-31ea-4ebd-bfcd-281c4ed16fc9 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:38.560292 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-0d201441-31ea-4ebd-bfcd-281c4ed16fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:38.702792 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 882/1737] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:38.713866 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23aabcd-96f8-453b-b9af-155ff06c67b2 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:38.715341 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-e23aabcd-96f8-453b-b9af-155ff06c67b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:38.794694 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:38.796857 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23aabcd-96f8-453b-b9af-155ff06c67b2 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:38.802802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1729c30d-aeaa-4360-a7bf-226a262ba500 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:38.807347 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729c30d-aeaa-4360-a7bf-226a262ba500 service neutron] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:38.807347 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729c30d-aeaa-4360-a7bf-226a262ba500 service neutron] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:38.807347 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729c30d-aeaa-4360-a7bf-226a262ba500 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:38.816587 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1729c30d-aeaa-4360-a7bf-226a262ba500 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:38.818642 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1729c30d-aeaa-4360-a7bf-226a262ba500 service neutron] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:38.836114 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 856/1738] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:12:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:12:38.855415 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:39.276989 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d201441-31ea-4ebd-bfcd-281c4ed16fc9 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:39.279294 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-0d201441-31ea-4ebd-bfcd-281c4ed16fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:39.294113 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d201441-31ea-4ebd-bfcd-281c4ed16fc9 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:39.302132 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0643a16-8873-4513-9a78-7910f4f32515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:39.305318 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0643a16-8873-4513-9a78-7910f4f32515 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:39.306130 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0643a16-8873-4513-9a78-7910f4f32515 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:39.307684 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0643a16-8873-4513-9a78-7910f4f32515 admin admin] PATH_INFO: `/v3/projects/318d93c0ba0f4501a5b7df9006720fc9` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:39.309859 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0643a16-8873-4513-9a78-7910f4f32515 admin admin] RBAC: Authorizing `identity:delete_project(project_id=318d93c0ba0f4501a5b7df9006720fc9)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:39.312423 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0643a16-8873-4513-9a78-7910f4f32515 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:39.469805 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1729c30d-aeaa-4360-a7bf-226a262ba500 service neutron] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:39.471501 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-1729c30d-aeaa-4360-a7bf-226a262ba500 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:39.494058 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1729c30d-aeaa-4360-a7bf-226a262ba500 service neutron] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:39.499101 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea538ae-611a-4925-9f08-eb6355dcac34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:39.499684 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea538ae-611a-4925-9f08-eb6355dcac34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:39.499966 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea538ae-611a-4925-9f08-eb6355dcac34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:39.500243 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea538ae-611a-4925-9f08-eb6355dcac34 admin admin] PATH_INFO: `/v3/projects/264d1e5aa00a4d5e9b51a39a5315737c` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:39.501364 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea538ae-611a-4925-9f08-eb6355dcac34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=264d1e5aa00a4d5e9b51a39a5315737c)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:39.503657 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea538ae-611a-4925-9f08-eb6355dcac34 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:39.563196 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-b0643a16-8873-4513-9a78-7910f4f32515 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '318d93c0ba0f4501a5b7df9006720fc9', 'request_id': 'req-b0643a16-8873-4513-9a78-7910f4f32515'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:39.579855 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-b0643a16-8873-4513-9a78-7910f4f32515 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '318d93c0ba0f4501a5b7df9006720fc9', 'request_id': 'req-b0643a16-8873-4513-9a78-7910f4f32515'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:39.611835 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:INSERT=5,keystone:SELECT=86,keystone:DELETE=9 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:39.683592 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-b0643a16-8873-4513-9a78-7910f4f32515 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '318d93c0ba0f4501a5b7df9006720fc9', 'request_id': 'req-b0643a16-8873-4513-9a78-7910f4f32515'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:39.709353 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-b0643a16-8873-4513-9a78-7910f4f32515 admin admin] The token cache is being invalidate because project 318d93c0ba0f4501a5b7df9006720fc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:12:39.709690 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-b0643a16-8873-4513-9a78-7910f4f32515 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:39.722085 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:39.722240 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-b0643a16-8873-4513-9a78-7910f4f32515 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:39.726058 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 883/1739] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:12:39 2022] DELETE /identity/v3/projects/318d93c0ba0f4501a5b7df9006720fc9 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:12:39.730090 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0643a16-8873-4513-9a78-7910f4f32515 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:39.734164 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-b0643a16-8873-4513-9a78-7910f4f32515 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:39.764027 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-eea538ae-611a-4925-9f08-eb6355dcac34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '264d1e5aa00a4d5e9b51a39a5315737c', 'request_id': 'req-eea538ae-611a-4925-9f08-eb6355dcac34'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:39.864308 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0643a16-8873-4513-9a78-7910f4f32515 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:39.880043 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-eea538ae-611a-4925-9f08-eb6355dcac34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '264d1e5aa00a4d5e9b51a39a5315737c', 'request_id': 'req-eea538ae-611a-4925-9f08-eb6355dcac34'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:39.892315 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-eea538ae-611a-4925-9f08-eb6355dcac34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '264d1e5aa00a4d5e9b51a39a5315737c', 'request_id': 'req-eea538ae-611a-4925-9f08-eb6355dcac34'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:39.910825 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-eea538ae-611a-4925-9f08-eb6355dcac34 admin admin] The token cache is being invalidate because project 264d1e5aa00a4d5e9b51a39a5315737c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81650) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:12:39.911484 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-eea538ae-611a-4925-9f08-eb6355dcac34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:39.915195 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-eea538ae-611a-4925-9f08-eb6355dcac34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:39.915195 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 857/1740] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:12:39 2022] DELETE /identity/v3/projects/264d1e5aa00a4d5e9b51a39a5315737c => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:12:39.918210 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea538ae-611a-4925-9f08-eb6355dcac34 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:39.921794 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-eea538ae-611a-4925-9f08-eb6355dcac34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:39.955733 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96f895b-6c98-4fa5-8b8c-5bb57b1850ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:39.957221 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f895b-6c98-4fa5-8b8c-5bb57b1850ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:39.957886 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f895b-6c98-4fa5-8b8c-5bb57b1850ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:39.958512 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f895b-6c98-4fa5-8b8c-5bb57b1850ea admin admin] PATH_INFO: `/v3/users/d0f28495791d42cf884897f318ff9abf` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:39.960878 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96f895b-6c98-4fa5-8b8c-5bb57b1850ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0f28495791d42cf884897f318ff9abf)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:39.963675 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96f895b-6c98-4fa5-8b8c-5bb57b1850ea admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:40.026176 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea538ae-611a-4925-9f08-eb6355dcac34 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:40.031641 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6536cac-27a2-44f5-990a-d4934a5299d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:40.035475 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6536cac-27a2-44f5-990a-d4934a5299d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:40.035475 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6536cac-27a2-44f5-990a-d4934a5299d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:40.035475 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6536cac-27a2-44f5-990a-d4934a5299d4 admin admin] PATH_INFO: `/v3/users/c585bef1854f4737a82bf39796263a61` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:40.035475 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6536cac-27a2-44f5-990a-d4934a5299d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c585bef1854f4737a82bf39796263a61)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:40.037029 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6536cac-27a2-44f5-990a-d4934a5299d4 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:40.188526 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-d96f895b-6c98-4fa5-8b8c-5bb57b1850ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0f28495791d42cf884897f318ff9abf'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:40.278955 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-d96f895b-6c98-4fa5-8b8c-5bb57b1850ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0f28495791d42cf884897f318ff9abf'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:40.279951 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-d96f895b-6c98-4fa5-8b8c-5bb57b1850ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0f28495791d42cf884897f318ff9abf'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:40.280576 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-d96f895b-6c98-4fa5-8b8c-5bb57b1850ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0f28495791d42cf884897f318ff9abf'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:40.293279 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-d96f895b-6c98-4fa5-8b8c-5bb57b1850ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0f28495791d42cf884897f318ff9abf'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:40.305071 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-d6536cac-27a2-44f5-990a-d4934a5299d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c585bef1854f4737a82bf39796263a61'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:40.319646 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 884/1741] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:12:39 2022] DELETE /identity/v3/users/d0f28495791d42cf884897f318ff9abf => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:12:40.323971 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-d6536cac-27a2-44f5-990a-d4934a5299d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c585bef1854f4737a82bf39796263a61'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:40.324737 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-d6536cac-27a2-44f5-990a-d4934a5299d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c585bef1854f4737a82bf39796263a61'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:40.410838 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-d6536cac-27a2-44f5-990a-d4934a5299d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c585bef1854f4737a82bf39796263a61'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:40.411533 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-d6536cac-27a2-44f5-990a-d4934a5299d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c585bef1854f4737a82bf39796263a61'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:40.425712 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 858/1742] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:12:39 2022] DELETE /identity/v3/users/c585bef1854f4737a82bf39796263a61 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:12:40.864017 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96f895b-6c98-4fa5-8b8c-5bb57b1850ea admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:40.865494 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-d96f895b-6c98-4fa5-8b8c-5bb57b1850ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:40.968465 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6536cac-27a2-44f5-990a-d4934a5299d4 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:40.970281 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-d6536cac-27a2-44f5-990a-d4934a5299d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:40.978983 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96f895b-6c98-4fa5-8b8c-5bb57b1850ea admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:41.086135 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8254e5b-261d-42fd-901f-96b0a263cd4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:41.086135 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8254e5b-261d-42fd-901f-96b0a263cd4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:41.086135 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8254e5b-261d-42fd-901f-96b0a263cd4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:41.086135 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8254e5b-261d-42fd-901f-96b0a263cd4a admin admin] PATH_INFO: `/v3/projects/76c0629228a943bb8f513f775b351495` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:41.086392 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8254e5b-261d-42fd-901f-96b0a263cd4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=76c0629228a943bb8f513f775b351495)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:41.086392 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8254e5b-261d-42fd-901f-96b0a263cd4a admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:41.110872 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:41.160581 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6536cac-27a2-44f5-990a-d4934a5299d4 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:41.165859 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4697320c-6964-4aba-bbe4-1423b1f80be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:41.166510 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4697320c-6964-4aba-bbe4-1423b1f80be5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:41.166789 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4697320c-6964-4aba-bbe4-1423b1f80be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:41.167104 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4697320c-6964-4aba-bbe4-1423b1f80be5 admin admin] PATH_INFO: `/v3/projects/9e0160d0266041cfab13e00bf8253412` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:41.168247 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4697320c-6964-4aba-bbe4-1423b1f80be5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e0160d0266041cfab13e00bf8253412)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:41.170200 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4697320c-6964-4aba-bbe4-1423b1f80be5 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:41.200269 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e8254e5b-261d-42fd-901f-96b0a263cd4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76c0629228a943bb8f513f775b351495', 'request_id': 'req-e8254e5b-261d-42fd-901f-96b0a263cd4a'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:41.206576 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e8254e5b-261d-42fd-901f-96b0a263cd4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76c0629228a943bb8f513f775b351495', 'request_id': 'req-e8254e5b-261d-42fd-901f-96b0a263cd4a'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:41.212060 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:41.280459 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-4697320c-6964-4aba-bbe4-1423b1f80be5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e0160d0266041cfab13e00bf8253412', 'request_id': 'req-4697320c-6964-4aba-bbe4-1423b1f80be5'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:41.312787 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e8254e5b-261d-42fd-901f-96b0a263cd4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76c0629228a943bb8f513f775b351495', 'request_id': 'req-e8254e5b-261d-42fd-901f-96b0a263cd4a'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:41.327845 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e8254e5b-261d-42fd-901f-96b0a263cd4a admin admin] The token cache is being invalidate because project 76c0629228a943bb8f513f775b351495 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:12:41.328135 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e8254e5b-261d-42fd-901f-96b0a263cd4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:41.328805 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e8254e5b-261d-42fd-901f-96b0a263cd4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:41.334020 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 885/1743] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:12:40 2022] DELETE /identity/v3/projects/76c0629228a943bb8f513f775b351495 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:12:41.336812 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a713c9-1994-4fd6-a8d4-1fea6cc0c22c None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:41.337099 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a713c9-1994-4fd6-a8d4-1fea6cc0c22c None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:41.337378 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a713c9-1994-4fd6-a8d4-1fea6cc0c22c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:41.361865 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-04a713c9-1994-4fd6-a8d4-1fea6cc0c22c None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:41.363131 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-04a713c9-1994-4fd6-a8d4-1fea6cc0c22c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:41.400056 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-4697320c-6964-4aba-bbe4-1423b1f80be5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e0160d0266041cfab13e00bf8253412', 'request_id': 'req-4697320c-6964-4aba-bbe4-1423b1f80be5'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:41.406301 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-4697320c-6964-4aba-bbe4-1423b1f80be5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e0160d0266041cfab13e00bf8253412', 'request_id': 'req-4697320c-6964-4aba-bbe4-1423b1f80be5'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:41.419588 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-4697320c-6964-4aba-bbe4-1423b1f80be5 admin admin] The token cache is being invalidate because project 9e0160d0266041cfab13e00bf8253412 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81650) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:12:41.420182 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-4697320c-6964-4aba-bbe4-1423b1f80be5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:41.421115 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-4697320c-6964-4aba-bbe4-1423b1f80be5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:41.423368 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 859/1744] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:12:40 2022] DELETE /identity/v3/projects/9e0160d0266041cfab13e00bf8253412 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:12:41.428384 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018b378-203b-4c02-a876-f8ed52c55be0 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:41.428831 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018b378-203b-4c02-a876-f8ed52c55be0 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:41.429199 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 886/1745] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:41.429513 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018b378-203b-4c02-a876-f8ed52c55be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:41.437027 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c483c-084e-4988-b132-f5e034e0bc34 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:41.437500 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c483c-084e-4988-b132-f5e034e0bc34 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:41.437910 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c483c-084e-4988-b132-f5e034e0bc34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:41.450479 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-c018b378-203b-4c02-a876-f8ed52c55be0 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:41.451736 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-c018b378-203b-4c02-a876-f8ed52c55be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:41.467481 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 860/1746] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:41.469156 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-e91c483c-084e-4988-b132-f5e034e0bc34 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:41.470399 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-e91c483c-084e-4988-b132-f5e034e0bc34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:41.472412 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449d572d-47a5-4310-89b6-93bae408026e None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:41.472676 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449d572d-47a5-4310-89b6-93bae408026e None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:41.472960 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449d572d-47a5-4310-89b6-93bae408026e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:41.480372 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 887/1747] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:41.488396 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91c483c-084e-4988-b132-f5e034e0bc34 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:41.497586 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-449d572d-47a5-4310-89b6-93bae408026e None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:41.497726 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91c483c-084e-4988-b132-f5e034e0bc34 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:41.499238 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-449d572d-47a5-4310-89b6-93bae408026e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:41.502169 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3493e5-b327-4a9e-a7a5-5ddcb727d1f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:41.502765 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3493e5-b327-4a9e-a7a5-5ddcb727d1f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:41.503151 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3493e5-b327-4a9e-a7a5-5ddcb727d1f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:41.503433 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3493e5-b327-4a9e-a7a5-5ddcb727d1f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:41.504068 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3493e5-b327-4a9e-a7a5-5ddcb727d1f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:12:41.504724 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3493e5-b327-4a9e-a7a5-5ddcb727d1f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:41.506058 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3493e5-b327-4a9e-a7a5-5ddcb727d1f0 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:41.507987 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 861/1748] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 17:12:41.512143 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449d572d-47a5-4310-89b6-93bae408026e None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:41.521934 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 888/1749] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:12:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 17:12:41.522859 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449d572d-47a5-4310-89b6-93bae408026e None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:41.524273 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3493e5-b327-4a9e-a7a5-5ddcb727d1f0 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:41.527317 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393812f2-625c-42d0-a885-4149aa74dc03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:41.528118 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393812f2-625c-42d0-a885-4149aa74dc03 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:41.528546 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393812f2-625c-42d0-a885-4149aa74dc03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:41.528962 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393812f2-625c-42d0-a885-4149aa74dc03 admin admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:41.529757 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393812f2-625c-42d0-a885-4149aa74dc03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:12:41.530224 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393812f2-625c-42d0-a885-4149aa74dc03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:41.532037 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393812f2-625c-42d0-a885-4149aa74dc03 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:41.536062 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3493e5-b327-4a9e-a7a5-5ddcb727d1f0 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:41.545636 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155cb2ef-8c52-4d6c-a934-cb3cc76cdf2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:41.547462 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 862/1750] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:12:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 17:12:41.555944 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155cb2ef-8c52-4d6c-a934-cb3cc76cdf2a admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:41.555944 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155cb2ef-8c52-4d6c-a934-cb3cc76cdf2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:41.555944 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155cb2ef-8c52-4d6c-a934-cb3cc76cdf2a admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:41.555944 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155cb2ef-8c52-4d6c-a934-cb3cc76cdf2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:41.555944 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155cb2ef-8c52-4d6c-a934-cb3cc76cdf2a admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:41.556952 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393812f2-625c-42d0-a885-4149aa74dc03 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:41.568316 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393812f2-625c-42d0-a885-4149aa74dc03 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:41.573761 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9288d3-150a-4fb1-a806-25e8187b0049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:41.574358 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9288d3-150a-4fb1-a806-25e8187b0049 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:41.574644 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9288d3-150a-4fb1-a806-25e8187b0049 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:41.574964 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9288d3-150a-4fb1-a806-25e8187b0049 admin admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:41.578595 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9288d3-150a-4fb1-a806-25e8187b0049 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:41.578595 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9288d3-150a-4fb1-a806-25e8187b0049 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:41.673419 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 889/1751] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:12:41 2022] POST /identity/v3/projects => generated 373 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:12:41.673613 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 863/1752] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:12:41 2022] POST /identity/v3/projects => generated 365 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 17:12:41.677629 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155cb2ef-8c52-4d6c-a934-cb3cc76cdf2a admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:41.680436 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:41.680841 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9288d3-150a-4fb1-a806-25e8187b0049 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:41.745100 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155cb2ef-8c52-4d6c-a934-cb3cc76cdf2a admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:41.750353 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eb5054-c8d6-475d-abc4-c42419305651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:41.751006 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb5054-c8d6-475d-abc4-c42419305651 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:41.751292 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb5054-c8d6-475d-abc4-c42419305651 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:41.751568 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb5054-c8d6-475d-abc4-c42419305651 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:41.752194 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eb5054-c8d6-475d-abc4-c42419305651 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:41.753318 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9288d3-150a-4fb1-a806-25e8187b0049 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:41.754391 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eb5054-c8d6-475d-abc4-c42419305651 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:41.759812 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad028daf-cc57-4e31-a9b5-fee19956fde4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:41.760440 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad028daf-cc57-4e31-a9b5-fee19956fde4 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:41.760724 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad028daf-cc57-4e31-a9b5-fee19956fde4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:41.760999 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad028daf-cc57-4e31-a9b5-fee19956fde4 admin admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:41.761632 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad028daf-cc57-4e31-a9b5-fee19956fde4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:41.764345 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad028daf-cc57-4e31-a9b5-fee19956fde4 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:41.846002 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 890/1753] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:12:41 2022] POST /identity/v3/users => generated 287 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 17:12:41.850283 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eb5054-c8d6-475d-abc4-c42419305651 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:41.867162 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eb5054-c8d6-475d-abc4-c42419305651 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:41.867162 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e10545-b9dd-4a11-b9e2-3e4ec112cd74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:41.867473 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e10545-b9dd-4a11-b9e2-3e4ec112cd74 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:41.867672 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e10545-b9dd-4a11-b9e2-3e4ec112cd74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:41.868327 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e10545-b9dd-4a11-b9e2-3e4ec112cd74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:41.869481 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e10545-b9dd-4a11-b9e2-3e4ec112cd74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:41.872044 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e10545-b9dd-4a11-b9e2-3e4ec112cd74 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:41.882084 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 864/1754] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:12:41 2022] POST /identity/v3/users => generated 283 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:12:41.885023 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad028daf-cc57-4e31-a9b5-fee19956fde4 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:41.887027 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 891/1755] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:12:41 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:12:41.887962 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e10545-b9dd-4a11-b9e2-3e4ec112cd74 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:41.895383 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad028daf-cc57-4e31-a9b5-fee19956fde4 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:41.897931 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e10545-b9dd-4a11-b9e2-3e4ec112cd74 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:41.901125 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b16f74e-4b42-4d9f-8469-9a1a537728d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:41.901740 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b16f74e-4b42-4d9f-8469-9a1a537728d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:41.902007 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b16f74e-4b42-4d9f-8469-9a1a537728d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:41.902278 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b16f74e-4b42-4d9f-8469-9a1a537728d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:41.903012 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b16f74e-4b42-4d9f-8469-9a1a537728d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:41.903112 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847bb611-f4f3-47db-88e5-45aecbadf781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:41.903808 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847bb611-f4f3-47db-88e5-45aecbadf781 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:41.904173 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847bb611-f4f3-47db-88e5-45aecbadf781 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:41.904520 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847bb611-f4f3-47db-88e5-45aecbadf781 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:41.904819 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b16f74e-4b42-4d9f-8469-9a1a537728d2 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:41.905340 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847bb611-f4f3-47db-88e5-45aecbadf781 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:41.907225 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847bb611-f4f3-47db-88e5-45aecbadf781 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:41.918945 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 865/1756] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:12:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 17:12:41.919644 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 892/1757] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:12:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:12:41.923859 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b16f74e-4b42-4d9f-8469-9a1a537728d2 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:41.924354 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847bb611-f4f3-47db-88e5-45aecbadf781 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:41.934500 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:41.936427 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847bb611-f4f3-47db-88e5-45aecbadf781 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:41.939004 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b16f74e-4b42-4d9f-8469-9a1a537728d2 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:41.941941 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cbb613-23be-4ab0-8ad8-32d52cc10d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:41.942724 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cbb613-23be-4ab0-8ad8-32d52cc10d41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:41.943133 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cbb613-23be-4ab0-8ad8-32d52cc10d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:41.943478 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cbb613-23be-4ab0-8ad8-32d52cc10d41 admin admin] PATH_INFO: `/v3/projects/637180141b1944859c22bb909e376e10/users/413962826e1843f68fc2bda00e67af18/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:41.944953 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff81644-da4b-484d-99f9-2dc0be085994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:41.945645 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff81644-da4b-484d-99f9-2dc0be085994 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:41.946008 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff81644-da4b-484d-99f9-2dc0be085994 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:41.946354 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff81644-da4b-484d-99f9-2dc0be085994 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:41.947232 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff81644-da4b-484d-99f9-2dc0be085994 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:41.949076 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff81644-da4b-484d-99f9-2dc0be085994 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:41.964672 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 866/1758] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:12:41 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:12:41.964863 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cbb613-23be-4ab0-8ad8-32d52cc10d41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=637180141b1944859c22bb909e376e10, user_id=413962826e1843f68fc2bda00e67af18, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:41.964863 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cbb613-23be-4ab0-8ad8-32d52cc10d41 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:41.967191 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff81644-da4b-484d-99f9-2dc0be085994 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:41.978492 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff81644-da4b-484d-99f9-2dc0be085994 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:41.983685 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c99259d-5ac7-4725-9d87-cc9537adde18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:41.984678 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99259d-5ac7-4725-9d87-cc9537adde18 admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:41.984881 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99259d-5ac7-4725-9d87-cc9537adde18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:41.985158 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99259d-5ac7-4725-9d87-cc9537adde18 admin admin] PATH_INFO: `/v3/projects/9e46398f6f034daaab53162018c771a4/users/b0013b653d7e4a06b1fc0ead54353d81/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:42.001859 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c99259d-5ac7-4725-9d87-cc9537adde18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e46398f6f034daaab53162018c771a4, user_id=b0013b653d7e4a06b1fc0ead54353d81, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:42.004480 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c99259d-5ac7-4725-9d87-cc9537adde18 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:42.049514 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 893/1759] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:12:41 2022] PUT /identity/v3/projects/637180141b1944859c22bb909e376e10/users/413962826e1843f68fc2bda00e67af18/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:12:42.063546 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc58679-3f37-4512-9045-36979ac4d229 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:42.063837 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc58679-3f37-4512-9045-36979ac4d229 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:42.064430 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc58679-3f37-4512-9045-36979ac4d229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:42.095563 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 867/1760] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:12:41 2022] PUT /identity/v3/projects/9e46398f6f034daaab53162018c771a4/users/b0013b653d7e4a06b1fc0ead54353d81/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:12:42.098690 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-7fc58679-3f37-4512-9045-36979ac4d229 None None] MFA Rules not processed for user `413962826e1843f68fc2bda00e67af18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:42.100593 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-7fc58679-3f37-4512-9045-36979ac4d229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:42.108877 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e2961-adb6-4647-9e64-caaac22af9e3 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:42.109357 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e2961-adb6-4647-9e64-caaac22af9e3 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:42.109634 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e2961-adb6-4647-9e64-caaac22af9e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:42.139731 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-4e8e2961-adb6-4647-9e64-caaac22af9e3 None None] MFA Rules not processed for user `b0013b653d7e4a06b1fc0ead54353d81`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:42.141971 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-4e8e2961-adb6-4647-9e64-caaac22af9e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:42.173190 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 894/1761] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:42.177413 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc58679-3f37-4512-9045-36979ac4d229 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:42.208317 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:42.253751 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc58679-3f37-4512-9045-36979ac4d229 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:42.258705 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e475ac-7f79-47a2-aa25-b7bb0e8ccae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:42.259537 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 868/1762] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:42 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 17:12:42.259651 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e475ac-7f79-47a2-aa25-b7bb0e8ccae2 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:42.259651 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e475ac-7f79-47a2-aa25-b7bb0e8ccae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:42.259945 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e475ac-7f79-47a2-aa25-b7bb0e8ccae2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:42.260631 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e475ac-7f79-47a2-aa25-b7bb0e8ccae2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:42.264407 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8e2961-adb6-4647-9e64-caaac22af9e3 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:42.265338 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e475ac-7f79-47a2-aa25-b7bb0e8ccae2 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:42.276595 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8e2961-adb6-4647-9e64-caaac22af9e3 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:42.282590 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e461af24-aac5-4fed-b0a2-5d5c67651631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:42.283316 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e461af24-aac5-4fed-b0a2-5d5c67651631 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:42.283823 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e461af24-aac5-4fed-b0a2-5d5c67651631 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:42.284133 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e461af24-aac5-4fed-b0a2-5d5c67651631 admin admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:42.284759 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e461af24-aac5-4fed-b0a2-5d5c67651631 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:42.286675 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e461af24-aac5-4fed-b0a2-5d5c67651631 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:42.371560 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 895/1763] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:12:42 2022] POST /identity/v3/projects => generated 373 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 17:12:42.375303 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e475ac-7f79-47a2-aa25-b7bb0e8ccae2 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:42.433525 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 869/1764] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:12:42 2022] POST /identity/v3/projects => generated 365 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:12:42.437718 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e461af24-aac5-4fed-b0a2-5d5c67651631 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:42.441379 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e475ac-7f79-47a2-aa25-b7bb0e8ccae2 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:42.452515 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:42.503167 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840b08dc-050a-48f3-be7b-452f4eab5235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:42.503167 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b08dc-050a-48f3-be7b-452f4eab5235 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:42.503167 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b08dc-050a-48f3-be7b-452f4eab5235 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:42.503167 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b08dc-050a-48f3-be7b-452f4eab5235 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:42.503481 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840b08dc-050a-48f3-be7b-452f4eab5235 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:42.503920 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840b08dc-050a-48f3-be7b-452f4eab5235 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:42.504122 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e461af24-aac5-4fed-b0a2-5d5c67651631 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:42.509206 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e00380d-d05d-4542-b5ec-cfa6a602b428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:42.509821 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e00380d-d05d-4542-b5ec-cfa6a602b428 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:42.510101 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e00380d-d05d-4542-b5ec-cfa6a602b428 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:42.510372 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e00380d-d05d-4542-b5ec-cfa6a602b428 admin admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:42.511018 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e00380d-d05d-4542-b5ec-cfa6a602b428 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:42.513275 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e00380d-d05d-4542-b5ec-cfa6a602b428 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:42.530923 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:42.586540 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 896/1765] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:12:42 2022] POST /identity/v3/users => generated 293 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 17:12:42.590531 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840b08dc-050a-48f3-be7b-452f4eab5235 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:42.602827 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 870/1766] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:12:42 2022] POST /identity/v3/users => generated 289 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 17:12:42.603014 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840b08dc-050a-48f3-be7b-452f4eab5235 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:42.604423 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e00380d-d05d-4542-b5ec-cfa6a602b428 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:42.607373 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8cc6f6-1b94-4612-aa3c-70708b471717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:42.607988 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8cc6f6-1b94-4612-aa3c-70708b471717 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:42.608274 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8cc6f6-1b94-4612-aa3c-70708b471717 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:42.608553 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8cc6f6-1b94-4612-aa3c-70708b471717 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:42.609195 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8cc6f6-1b94-4612-aa3c-70708b471717 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:42.610804 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8cc6f6-1b94-4612-aa3c-70708b471717 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:42.614602 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e00380d-d05d-4542-b5ec-cfa6a602b428 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:42.620026 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb48303-1bec-4215-b195-0765de5ace60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:42.623553 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb48303-1bec-4215-b195-0765de5ace60 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:42.623553 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb48303-1bec-4215-b195-0765de5ace60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:42.623553 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb48303-1bec-4215-b195-0765de5ace60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:42.623876 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 897/1767] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:12:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 17:12:42.623963 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb48303-1bec-4215-b195-0765de5ace60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:42.623963 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb48303-1bec-4215-b195-0765de5ace60 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:42.626572 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8cc6f6-1b94-4612-aa3c-70708b471717 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:42.637635 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 871/1768] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:12:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 17:12:42.637798 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8cc6f6-1b94-4612-aa3c-70708b471717 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:42.638830 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb48303-1bec-4215-b195-0765de5ace60 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:42.641571 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3386fb72-a431-4848-896a-aa430525b77a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:42.642253 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3386fb72-a431-4848-896a-aa430525b77a admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:42.642529 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3386fb72-a431-4848-896a-aa430525b77a admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:42.642803 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3386fb72-a431-4848-896a-aa430525b77a admin admin] PATH_INFO: `/v3/projects/66b0ad02ff55423582b2c69824416954/users/cdb883bd438a460ea89ad3bd277df2fe/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:42.649515 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb48303-1bec-4215-b195-0765de5ace60 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:42.654119 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6c393d-dd7c-4a3f-8c35-39c9d537c7ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:42.654766 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c393d-dd7c-4a3f-8c35-39c9d537c7ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:42.655152 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c393d-dd7c-4a3f-8c35-39c9d537c7ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:42.655427 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c393d-dd7c-4a3f-8c35-39c9d537c7ac admin admin] PATH_INFO: `/v3/projects/0e4973f62a6f4c13a7d66fdcb2fb64c3/users/db00dca6a8ca4c15bd0ecfc7a0d35e64/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:42.658375 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3386fb72-a431-4848-896a-aa430525b77a admin admin] RBAC: Authorizing `identity:create_grant(project_id=66b0ad02ff55423582b2c69824416954, user_id=cdb883bd438a460ea89ad3bd277df2fe, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:42.661059 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3386fb72-a431-4848-896a-aa430525b77a admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:42.671484 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6c393d-dd7c-4a3f-8c35-39c9d537c7ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e4973f62a6f4c13a7d66fdcb2fb64c3, user_id=db00dca6a8ca4c15bd0ecfc7a0d35e64, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:42.674376 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6c393d-dd7c-4a3f-8c35-39c9d537c7ac admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:42.734371 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 898/1769] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:12:42 2022] PUT /identity/v3/projects/66b0ad02ff55423582b2c69824416954/users/cdb883bd438a460ea89ad3bd277df2fe/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:12:42.738389 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3386fb72-a431-4848-896a-aa430525b77a admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:42.761762 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:42.766779 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 872/1770] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:12:42 2022] PUT /identity/v3/projects/0e4973f62a6f4c13a7d66fdcb2fb64c3/users/db00dca6a8ca4c15bd0ecfc7a0d35e64/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:12:42.769952 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6c393d-dd7c-4a3f-8c35-39c9d537c7ac admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:42.806864 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3386fb72-a431-4848-896a-aa430525b77a admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:42.840123 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6c393d-dd7c-4a3f-8c35-39c9d537c7ac admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:42.844721 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a641ed-278c-4ca8-8555-ef3a27ee62f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:42.845333 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a641ed-278c-4ca8-8555-ef3a27ee62f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:42.845614 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a641ed-278c-4ca8-8555-ef3a27ee62f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:42.845886 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a641ed-278c-4ca8-8555-ef3a27ee62f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:42.846587 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a641ed-278c-4ca8-8555-ef3a27ee62f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:42.848552 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a641ed-278c-4ca8-8555-ef3a27ee62f4 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:42.859444 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 873/1771] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:12:42 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:12:42.863614 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a641ed-278c-4ca8-8555-ef3a27ee62f4 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:42.865508 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427d824c-d1e2-4ab9-8ba4-cfc0327f18af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:42.866439 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d824c-d1e2-4ab9-8ba4-cfc0327f18af admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:42.866739 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d824c-d1e2-4ab9-8ba4-cfc0327f18af admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:42.867399 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d824c-d1e2-4ab9-8ba4-cfc0327f18af admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:42.868111 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427d824c-d1e2-4ab9-8ba4-cfc0327f18af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:42.869797 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427d824c-d1e2-4ab9-8ba4-cfc0327f18af admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:42.873762 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a641ed-278c-4ca8-8555-ef3a27ee62f4 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:42.879977 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713d6d71-d4ac-4c7b-8ae5-d94a3b12b8f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:42.880415 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d6d71-d4ac-4c7b-8ae5-d94a3b12b8f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:42.880699 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d6d71-d4ac-4c7b-8ae5-d94a3b12b8f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:42.880973 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d6d71-d4ac-4c7b-8ae5-d94a3b12b8f8 admin admin] PATH_INFO: `/v3/projects/0e4973f62a6f4c13a7d66fdcb2fb64c3/users/db00dca6a8ca4c15bd0ecfc7a0d35e64/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:42.883991 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 899/1772] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:12:42 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 17:12:42.884779 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713d6d71-d4ac-4c7b-8ae5-d94a3b12b8f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e4973f62a6f4c13a7d66fdcb2fb64c3, user_id=db00dca6a8ca4c15bd0ecfc7a0d35e64, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:42.884895 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427d824c-d1e2-4ab9-8ba4-cfc0327f18af admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:42.887322 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713d6d71-d4ac-4c7b-8ae5-d94a3b12b8f8 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:42.894725 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427d824c-d1e2-4ab9-8ba4-cfc0327f18af admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:42.900308 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 874/1773] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:12:42 2022] PUT /identity/v3/projects/0e4973f62a6f4c13a7d66fdcb2fb64c3/users/db00dca6a8ca4c15bd0ecfc7a0d35e64/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:12:42.910476 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900cd851-7449-4f4a-84a9-913dac453034 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:42.910766 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900cd851-7449-4f4a-84a9-913dac453034 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:42.911088 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900cd851-7449-4f4a-84a9-913dac453034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:42.941432 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-900cd851-7449-4f4a-84a9-913dac453034 None None] MFA Rules not processed for user `db00dca6a8ca4c15bd0ecfc7a0d35e64`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:42.942053 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:42.942671 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-900cd851-7449-4f4a-84a9-913dac453034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:42.961039 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ecc47b-95c6-4951-a4a8-558549b6dd23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:42.961715 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ecc47b-95c6-4951-a4a8-558549b6dd23 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:42.962000 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ecc47b-95c6-4951-a4a8-558549b6dd23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:42.962275 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ecc47b-95c6-4951-a4a8-558549b6dd23 admin admin] PATH_INFO: `/v3/projects/66b0ad02ff55423582b2c69824416954/users/cdb883bd438a460ea89ad3bd277df2fe/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:42.964562 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ecc47b-95c6-4951-a4a8-558549b6dd23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66b0ad02ff55423582b2c69824416954, user_id=cdb883bd438a460ea89ad3bd277df2fe, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:42.967374 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ecc47b-95c6-4951-a4a8-558549b6dd23 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:42.978755 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 900/1774] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:12:42 2022] PUT /identity/v3/projects/66b0ad02ff55423582b2c69824416954/users/cdb883bd438a460ea89ad3bd277df2fe/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:12:42.987252 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:42.993164 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdbb93a-be7f-4d41-993c-cf302aead5f4 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:42.993164 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdbb93a-be7f-4d41-993c-cf302aead5f4 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:42.993164 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdbb93a-be7f-4d41-993c-cf302aead5f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:43.022686 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-1fdbb93a-be7f-4d41-993c-cf302aead5f4 None None] MFA Rules not processed for user `cdb883bd438a460ea89ad3bd277df2fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:43.022869 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 875/1775] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:42 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 17:12:43.023001 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-1fdbb93a-be7f-4d41-993c-cf302aead5f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:43.030890 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900cd851-7449-4f4a-84a9-913dac453034 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:43.032371 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-900cd851-7449-4f4a-84a9-913dac453034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:43.091566 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 901/1776] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:42 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 17:12:43.092815 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900cd851-7449-4f4a-84a9-913dac453034 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:43.102826 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdbb93a-be7f-4d41-993c-cf302aead5f4 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:43.104262 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-1fdbb93a-be7f-4d41-993c-cf302aead5f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:43.146610 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2faf571-c433-4eec-b1d0-3990f2e7a308 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:43.147332 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2faf571-c433-4eec-b1d0-3990f2e7a308 service cinder] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:43.147798 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2faf571-c433-4eec-b1d0-3990f2e7a308 service cinder] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:43.148083 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2faf571-c433-4eec-b1d0-3990f2e7a308 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:43.167759 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdbb93a-be7f-4d41-993c-cf302aead5f4 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:43.171415 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e03dd4f-64de-4953-87e8-1c16c9896c63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:43.172282 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e03dd4f-64de-4953-87e8-1c16c9896c63 service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:43.172566 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e03dd4f-64de-4953-87e8-1c16c9896c63 service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:43.172840 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e03dd4f-64de-4953-87e8-1c16c9896c63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:43.182307 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e03dd4f-64de-4953-87e8-1c16c9896c63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:43.183797 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e03dd4f-64de-4953-87e8-1c16c9896c63 service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:43.193330 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 902/1777] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:12:43 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 17:12:43.210659 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2faf571-c433-4eec-b1d0-3990f2e7a308 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:43.212162 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2faf571-c433-4eec-b1d0-3990f2e7a308 service cinder] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:43.221688 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 876/1778] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:12:43 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 17:12:43.444901 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e03dd4f-64de-4953-87e8-1c16c9896c63 service cinder] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:43.451532 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e03dd4f-64de-4953-87e8-1c16c9896c63 service cinder] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:43.457174 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2faf571-c433-4eec-b1d0-3990f2e7a308 service cinder] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:43.468787 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2faf571-c433-4eec-b1d0-3990f2e7a308 service cinder] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:43.471379 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b8897f-9c99-44be-afa4-b79f1a2c9723 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:43.472143 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b8897f-9c99-44be-afa4-b79f1a2c9723 service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:43.472301 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b8897f-9c99-44be-afa4-b79f1a2c9723 service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:43.474558 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b8897f-9c99-44be-afa4-b79f1a2c9723 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:43.474804 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e9a733-f9e2-475b-ae5e-31a766ed1b61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:43.474804 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e9a733-f9e2-475b-ae5e-31a766ed1b61 service cinder] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:43.474804 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e9a733-f9e2-475b-ae5e-31a766ed1b61 service cinder] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:43.474804 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e9a733-f9e2-475b-ae5e-31a766ed1b61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:43.511826 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:43.543550 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e9a733-f9e2-475b-ae5e-31a766ed1b61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:43.545058 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e9a733-f9e2-475b-ae5e-31a766ed1b61 service cinder] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:43.546290 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b8897f-9c99-44be-afa4-b79f1a2c9723 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:43.548069 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b8897f-9c99-44be-afa4-b79f1a2c9723 service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:43.555670 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 877/1779] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:12:43 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 17:12:43.558059 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 903/1780] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:12:43 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:12:45.083622 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e9a733-f9e2-475b-ae5e-31a766ed1b61 service cinder] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:45.090043 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e9a733-f9e2-475b-ae5e-31a766ed1b61 service cinder] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:45.097975 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d841aea2-4e34-40c0-86e6-5894ebbdd557 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:45.098617 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d841aea2-4e34-40c0-86e6-5894ebbdd557 service cinder] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:45.101201 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d841aea2-4e34-40c0-86e6-5894ebbdd557 service cinder] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:45.106141 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d841aea2-4e34-40c0-86e6-5894ebbdd557 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:45.122369 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-d841aea2-4e34-40c0-86e6-5894ebbdd557 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:45.176047 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d4e52-6880-4fc8-af9d-6c4b8e5750f0 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:45.176670 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d4e52-6880-4fc8-af9d-6c4b8e5750f0 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:45.176670 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d4e52-6880-4fc8-af9d-6c4b8e5750f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:45.183689 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:45.258797 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d841aea2-4e34-40c0-86e6-5894ebbdd557 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:45.260614 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d841aea2-4e34-40c0-86e6-5894ebbdd557 service cinder] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:45.279091 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-f98d4e52-6880-4fc8-af9d-6c4b8e5750f0 None None] MFA Rules not processed for user `5de19a54ed104bcdb35ee4e5e793f1e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:45.280914 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-f98d4e52-6880-4fc8-af9d-6c4b8e5750f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:45.283891 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 878/1781] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:12:45 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:12:45.354335 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 904/1782] 10.222.0.82 () {58 vars in 1009 bytes} [Thu Aug 18 17:12:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:45.833131 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d841aea2-4e34-40c0-86e6-5894ebbdd557 service cinder] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:45.834994 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-d841aea2-4e34-40c0-86e6-5894ebbdd557 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:45.970832 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d841aea2-4e34-40c0-86e6-5894ebbdd557 service cinder] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:45.976843 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3350b283-1739-4ed4-bb94-b6cea718357e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1381673f81584f4b8084823e3aa99e2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:45.977644 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3350b283-1739-4ed4-bb94-b6cea718357e service swift] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:45.978072 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3350b283-1739-4ed4-bb94-b6cea718357e service swift] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:45.978365 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3350b283-1739-4ed4-bb94-b6cea718357e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:46.006572 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3350b283-1739-4ed4-bb94-b6cea718357e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:46.009946 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3350b283-1739-4ed4-bb94-b6cea718357e service swift] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:46.033777 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 879/1783] 10.222.0.82 () {60 vars in 1384 bytes} [Thu Aug 18 17:12:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 18 17:12:46.047722 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a903a6-0da2-4fc2-87bd-fe2714b41ff6 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:46.048040 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a903a6-0da2-4fc2-87bd-fe2714b41ff6 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:46.048325 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a903a6-0da2-4fc2-87bd-fe2714b41ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:46.075906 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-63a903a6-0da2-4fc2-87bd-fe2714b41ff6 None None] MFA Rules not processed for user `fe2403ede53148b2a113ffa2a605e987`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:46.077126 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-63a903a6-0da2-4fc2-87bd-fe2714b41ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:46.230756 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 905/1784] 10.222.0.83 () {58 vars in 1015 bytes} [Thu Aug 18 17:12:46 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:46.243342 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3350b283-1739-4ed4-bb94-b6cea718357e service swift] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:46.246297 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-3350b283-1739-4ed4-bb94-b6cea718357e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:46.290972 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3350b283-1739-4ed4-bb94-b6cea718357e service swift] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:46.295419 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef91af31-3670-4874-82f6-b52119c6cb6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:46.296219 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef91af31-3670-4874-82f6-b52119c6cb6d service neutron] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:46.296795 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef91af31-3670-4874-82f6-b52119c6cb6d service neutron] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:46.297390 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef91af31-3670-4874-82f6-b52119c6cb6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:46.307428 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef91af31-3670-4874-82f6-b52119c6cb6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:46.309614 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef91af31-3670-4874-82f6-b52119c6cb6d service neutron] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:46.322296 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 880/1785] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:12:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:12:47.666177 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a903a6-0da2-4fc2-87bd-fe2714b41ff6 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:47.667807 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-63a903a6-0da2-4fc2-87bd-fe2714b41ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:47.790284 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a903a6-0da2-4fc2-87bd-fe2714b41ff6 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:47.797163 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea57d537-1203-4ae2-bd89-01ede0601204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:47.798081 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea57d537-1203-4ae2-bd89-01ede0601204 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:47.798755 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea57d537-1203-4ae2-bd89-01ede0601204 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:47.799119 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea57d537-1203-4ae2-bd89-01ede0601204 admin admin] PATH_INFO: `/v3/users/1db2791b42554d468e3bae04b0253ace` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:47.800334 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea57d537-1203-4ae2-bd89-01ede0601204 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1db2791b42554d468e3bae04b0253ace)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:47.802540 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea57d537-1203-4ae2-bd89-01ede0601204 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:47.954170 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:48.076095 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ea57d537-1203-4ae2-bd89-01ede0601204 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1db2791b42554d468e3bae04b0253ace'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:48.177358 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ea57d537-1203-4ae2-bd89-01ede0601204 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1db2791b42554d468e3bae04b0253ace'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:48.180032 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ea57d537-1203-4ae2-bd89-01ede0601204 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1db2791b42554d468e3bae04b0253ace'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:48.182439 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ea57d537-1203-4ae2-bd89-01ede0601204 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1db2791b42554d468e3bae04b0253ace'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:48.193020 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ea57d537-1203-4ae2-bd89-01ede0601204 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1db2791b42554d468e3bae04b0253ace'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:48.267641 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 906/1786] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:12:47 2022] DELETE /identity/v3/users/1db2791b42554d468e3bae04b0253ace => generated 0 bytes in 605 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:12:48.272253 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994930c5-b825-4a66-afb9-8bc57bf09004 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:48.273215 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994930c5-b825-4a66-afb9-8bc57bf09004 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:48.273665 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994930c5-b825-4a66-afb9-8bc57bf09004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:48.317773 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-994930c5-b825-4a66-afb9-8bc57bf09004 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:48.321726 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-994930c5-b825-4a66-afb9-8bc57bf09004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:48.356295 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:48.414097 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 881/1787] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 17:12:48.426548 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea57d537-1203-4ae2-bd89-01ede0601204 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:48.427485 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-ea57d537-1203-4ae2-bd89-01ede0601204 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:48.535153 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea57d537-1203-4ae2-bd89-01ede0601204 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:48.542599 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e140fc13-cfca-482b-ad26-cd181aae54ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:48.543261 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e140fc13-cfca-482b-ad26-cd181aae54ba service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:48.551520 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e140fc13-cfca-482b-ad26-cd181aae54ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:48.552360 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e140fc13-cfca-482b-ad26-cd181aae54ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:48.577057 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e140fc13-cfca-482b-ad26-cd181aae54ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:48.580551 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e140fc13-cfca-482b-ad26-cd181aae54ba service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:48.594040 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee161094-ed93-4f2e-9817-9146997e9840 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:48.594319 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee161094-ed93-4f2e-9817-9146997e9840 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:48.594761 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee161094-ed93-4f2e-9817-9146997e9840 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:48.596144 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 882/1788] 10.222.0.82 () {54 vars in 921 bytes} [Thu Aug 18 17:12:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 17:12:48.606122 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 907/1789] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:12:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:12:48.627714 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15465c13-06fa-4af5-974e-2fb0305be607 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:48.640841 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15465c13-06fa-4af5-974e-2fb0305be607 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:48.640841 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15465c13-06fa-4af5-974e-2fb0305be607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:48.679516 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-15465c13-06fa-4af5-974e-2fb0305be607 None None] MFA Rules not processed for user `55723c756cae4e7ab636054db48de02d`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:48.682335 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-15465c13-06fa-4af5-974e-2fb0305be607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:48.861554 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 883/1790] 10.222.0.82 () {58 vars in 1018 bytes} [Thu Aug 18 17:12:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:48.903694 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e140fc13-cfca-482b-ad26-cd181aae54ba service neutron] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:48.905211 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-e140fc13-cfca-482b-ad26-cd181aae54ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:48.942561 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e140fc13-cfca-482b-ad26-cd181aae54ba service neutron] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:48.962463 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b51f55-bb46-48ff-812c-21153687cb25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55723c756cae4e7ab636054db48de02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:48.969703 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b51f55-bb46-48ff-812c-21153687cb25 service nova] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:48.969703 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b51f55-bb46-48ff-812c-21153687cb25 service nova] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:48.969703 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b51f55-bb46-48ff-812c-21153687cb25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:48.983450 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b51f55-bb46-48ff-812c-21153687cb25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:48.985039 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b51f55-bb46-48ff-812c-21153687cb25 service nova] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:49.020319 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 908/1791] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:12:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:12:49.577765 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15465c13-06fa-4af5-974e-2fb0305be607 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:49.579324 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-15465c13-06fa-4af5-974e-2fb0305be607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:49.595122 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15465c13-06fa-4af5-974e-2fb0305be607 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:49.603041 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5185496-ab2f-4dc3-91c0-9e873345ea43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:49.603699 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5185496-ab2f-4dc3-91c0-9e873345ea43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:49.603974 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5185496-ab2f-4dc3-91c0-9e873345ea43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:49.604481 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5185496-ab2f-4dc3-91c0-9e873345ea43 admin admin] PATH_INFO: `/v3/projects/07918f455d114a719fd4b7742348e1b9` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:49.605648 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5185496-ab2f-4dc3-91c0-9e873345ea43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07918f455d114a719fd4b7742348e1b9)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:49.607624 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5185496-ab2f-4dc3-91c0-9e873345ea43 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:49.750028 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-e5185496-ab2f-4dc3-91c0-9e873345ea43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07918f455d114a719fd4b7742348e1b9', 'request_id': 'req-e5185496-ab2f-4dc3-91c0-9e873345ea43'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:49.856428 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-e5185496-ab2f-4dc3-91c0-9e873345ea43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07918f455d114a719fd4b7742348e1b9', 'request_id': 'req-e5185496-ab2f-4dc3-91c0-9e873345ea43'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:49.862852 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-e5185496-ab2f-4dc3-91c0-9e873345ea43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07918f455d114a719fd4b7742348e1b9', 'request_id': 'req-e5185496-ab2f-4dc3-91c0-9e873345ea43'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:49.875183 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-e5185496-ab2f-4dc3-91c0-9e873345ea43 admin admin] The token cache is being invalidate because project 07918f455d114a719fd4b7742348e1b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81650) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:12:49.875475 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-e5185496-ab2f-4dc3-91c0-9e873345ea43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:49.876174 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-e5185496-ab2f-4dc3-91c0-9e873345ea43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:49.877665 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 884/1792] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:12:49 2022] DELETE /identity/v3/projects/07918f455d114a719fd4b7742348e1b9 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:12:49.882937 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ff410-fdb9-4cd8-b01d-05e567589817 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:49.883419 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ff410-fdb9-4cd8-b01d-05e567589817 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:49.883726 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ff410-fdb9-4cd8-b01d-05e567589817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:49.913579 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-ee4ff410-fdb9-4cd8-b01d-05e567589817 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:49.915976 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-ee4ff410-fdb9-4cd8-b01d-05e567589817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:49.992812 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 909/1793] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 17:12:50.001424 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e8e9fb-184f-490e-b791-b6ebcb06f174 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:50.001718 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e8e9fb-184f-490e-b791-b6ebcb06f174 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:50.001997 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e8e9fb-184f-490e-b791-b6ebcb06f174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:50.023217 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-98e8e9fb-184f-490e-b791-b6ebcb06f174 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:50.024449 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-98e8e9fb-184f-490e-b791-b6ebcb06f174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:50.034202 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 885/1794] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:50.038461 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4ff410-fdb9-4cd8-b01d-05e567589817 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:50.052405 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4ff410-fdb9-4cd8-b01d-05e567589817 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:50.055104 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f948d99f-2452-464d-819d-e0e79d292de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:50.055749 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f948d99f-2452-464d-819d-e0e79d292de9 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:50.055989 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f948d99f-2452-464d-819d-e0e79d292de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:50.056261 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f948d99f-2452-464d-819d-e0e79d292de9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:50.056906 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f948d99f-2452-464d-819d-e0e79d292de9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:12:50.057207 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f948d99f-2452-464d-819d-e0e79d292de9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:50.058838 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f948d99f-2452-464d-819d-e0e79d292de9 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:50.074076 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 910/1795] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:12:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 17:12:50.083918 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e8e9fb-184f-490e-b791-b6ebcb06f174 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:50.093738 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e8e9fb-184f-490e-b791-b6ebcb06f174 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:50.098429 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db068da-146e-4565-856d-1a64e9df3880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:50.099168 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db068da-146e-4565-856d-1a64e9df3880 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:50.099633 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db068da-146e-4565-856d-1a64e9df3880 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:50.100269 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db068da-146e-4565-856d-1a64e9df3880 admin admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:50.101161 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db068da-146e-4565-856d-1a64e9df3880 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:50.103404 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db068da-146e-4565-856d-1a64e9df3880 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:50.192538 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 886/1796] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:12:50 2022] POST /identity/v3/projects => generated 365 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:12:50.197366 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f948d99f-2452-464d-819d-e0e79d292de9 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:50.235471 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f948d99f-2452-464d-819d-e0e79d292de9 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:50.280511 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:50.340504 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a83637-6fef-469e-a560-bd3ebfd01a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:50.341547 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a83637-6fef-469e-a560-bd3ebfd01a33 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:50.342765 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a83637-6fef-469e-a560-bd3ebfd01a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:50.343373 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a83637-6fef-469e-a560-bd3ebfd01a33 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:50.344369 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a83637-6fef-469e-a560-bd3ebfd01a33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:50.347483 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a83637-6fef-469e-a560-bd3ebfd01a33 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:50.512965 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 911/1797] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:12:50 2022] POST /identity/v3/users => generated 283 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:12:50.517144 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db068da-146e-4565-856d-1a64e9df3880 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:50.528729 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db068da-146e-4565-856d-1a64e9df3880 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:50.543242 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d07b7c2-bc1e-45b8-889a-5a9fd76c4f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:50.543242 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d07b7c2-bc1e-45b8-889a-5a9fd76c4f3b admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:50.543242 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d07b7c2-bc1e-45b8-889a-5a9fd76c4f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:50.543242 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d07b7c2-bc1e-45b8-889a-5a9fd76c4f3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:50.543609 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d07b7c2-bc1e-45b8-889a-5a9fd76c4f3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:50.543609 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d07b7c2-bc1e-45b8-889a-5a9fd76c4f3b admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:50.543609 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:50.547945 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 887/1798] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:12:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:12:50.551981 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a83637-6fef-469e-a560-bd3ebfd01a33 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:50.563766 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a83637-6fef-469e-a560-bd3ebfd01a33 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:50.568607 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff455acd-45e3-4be7-bfb0-5c85561df3e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:50.569226 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff455acd-45e3-4be7-bfb0-5c85561df3e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:50.569511 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff455acd-45e3-4be7-bfb0-5c85561df3e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:50.569785 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff455acd-45e3-4be7-bfb0-5c85561df3e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:50.570435 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff455acd-45e3-4be7-bfb0-5c85561df3e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:50.572233 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff455acd-45e3-4be7-bfb0-5c85561df3e2 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:50.584440 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 912/1799] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:12:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:12:50.588888 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d07b7c2-bc1e-45b8-889a-5a9fd76c4f3b admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:50.601729 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d07b7c2-bc1e-45b8-889a-5a9fd76c4f3b admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:50.608848 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec71a15e-9292-4f4d-85a2-7f1525987895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:50.609534 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec71a15e-9292-4f4d-85a2-7f1525987895 admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:50.609815 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec71a15e-9292-4f4d-85a2-7f1525987895 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:50.610090 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec71a15e-9292-4f4d-85a2-7f1525987895 admin admin] PATH_INFO: `/v3/projects/15a3cdf559004e2986b5415735dff20d/users/76ea109f1fc54723a59a8f40d435d4e7/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:50.627748 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec71a15e-9292-4f4d-85a2-7f1525987895 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15a3cdf559004e2986b5415735dff20d, user_id=76ea109f1fc54723a59a8f40d435d4e7, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:50.630805 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec71a15e-9292-4f4d-85a2-7f1525987895 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:50.744041 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 888/1800] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:12:50 2022] PUT /identity/v3/projects/15a3cdf559004e2986b5415735dff20d/users/76ea109f1fc54723a59a8f40d435d4e7/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:12:50.757651 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424ed83-c1e5-4a0f-991b-33d023289393 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:50.758024 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424ed83-c1e5-4a0f-991b-33d023289393 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:50.758360 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424ed83-c1e5-4a0f-991b-33d023289393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:50.798661 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-a424ed83-c1e5-4a0f-991b-33d023289393 None None] MFA Rules not processed for user `76ea109f1fc54723a59a8f40d435d4e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:50.802111 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-a424ed83-c1e5-4a0f-991b-33d023289393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:50.868352 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 913/1801] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:50 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 17:12:50.878596 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec71a15e-9292-4f4d-85a2-7f1525987895 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:50.951035 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec71a15e-9292-4f4d-85a2-7f1525987895 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:50.955985 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f94c7d8-25e7-45db-9c37-c4b3380aa5ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:50.956605 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f94c7d8-25e7-45db-9c37-c4b3380aa5ce admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:50.957099 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f94c7d8-25e7-45db-9c37-c4b3380aa5ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:50.957278 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f94c7d8-25e7-45db-9c37-c4b3380aa5ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:50.957884 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f94c7d8-25e7-45db-9c37-c4b3380aa5ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:50.960288 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f94c7d8-25e7-45db-9c37-c4b3380aa5ce admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:51.082130 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 889/1802] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:12:50 2022] POST /identity/v3/projects => generated 365 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:12:51.083455 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a424ed83-c1e5-4a0f-991b-33d023289393 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:51.098805 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a424ed83-c1e5-4a0f-991b-33d023289393 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:51.165420 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5288d8-3fa7-4d2b-89cc-5b9da87a51ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:51.166039 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5288d8-3fa7-4d2b-89cc-5b9da87a51ac admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:51.166323 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5288d8-3fa7-4d2b-89cc-5b9da87a51ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:51.166691 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5288d8-3fa7-4d2b-89cc-5b9da87a51ac admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:51.167484 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5288d8-3fa7-4d2b-89cc-5b9da87a51ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:51.169405 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5288d8-3fa7-4d2b-89cc-5b9da87a51ac admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:51.273316 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 914/1803] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:12:51 2022] POST /identity/v3/users => generated 289 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:12:51.276273 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f94c7d8-25e7-45db-9c37-c4b3380aa5ce admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:51.294222 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f94c7d8-25e7-45db-9c37-c4b3380aa5ce admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:51.300953 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec955bfe-6c07-4ae1-bbf6-b3153cca9224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:51.301877 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec955bfe-6c07-4ae1-bbf6-b3153cca9224 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:51.303171 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec955bfe-6c07-4ae1-bbf6-b3153cca9224 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:51.303940 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec955bfe-6c07-4ae1-bbf6-b3153cca9224 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:51.304831 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec955bfe-6c07-4ae1-bbf6-b3153cca9224 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:51.306715 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec955bfe-6c07-4ae1-bbf6-b3153cca9224 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:51.325217 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 890/1804] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:12:51 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:12:51.330334 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5288d8-3fa7-4d2b-89cc-5b9da87a51ac admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:51.346091 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5288d8-3fa7-4d2b-89cc-5b9da87a51ac admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:51.358936 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72933c0-5135-4012-a8ed-d7e5b2dc0e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:51.362672 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72933c0-5135-4012-a8ed-d7e5b2dc0e5e admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:51.362672 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72933c0-5135-4012-a8ed-d7e5b2dc0e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:51.362672 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72933c0-5135-4012-a8ed-d7e5b2dc0e5e admin admin] PATH_INFO: `/v3/projects/70af716b8d7d4f5583e0370c1a9b4238/users/ce8c2c602fac4ca791762f74973b3508/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:51.384563 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72933c0-5135-4012-a8ed-d7e5b2dc0e5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=70af716b8d7d4f5583e0370c1a9b4238, user_id=ce8c2c602fac4ca791762f74973b3508, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:51.388115 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72933c0-5135-4012-a8ed-d7e5b2dc0e5e admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:51.492901 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 915/1805] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:12:51 2022] PUT /identity/v3/projects/70af716b8d7d4f5583e0370c1a9b4238/users/ce8c2c602fac4ca791762f74973b3508/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:12:51.497352 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec955bfe-6c07-4ae1-bbf6-b3153cca9224 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:51.509395 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec955bfe-6c07-4ae1-bbf6-b3153cca9224 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:51.624320 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fb60af-a51e-4895-b385-6682854c0b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:51.624996 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb60af-a51e-4895-b385-6682854c0b75 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:51.625218 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb60af-a51e-4895-b385-6682854c0b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:51.625493 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb60af-a51e-4895-b385-6682854c0b75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:51.626349 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fb60af-a51e-4895-b385-6682854c0b75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:51.627998 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fb60af-a51e-4895-b385-6682854c0b75 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:51.639450 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 891/1806] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:12:51 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:12:51.643714 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72933c0-5135-4012-a8ed-d7e5b2dc0e5e admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:51.654187 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72933c0-5135-4012-a8ed-d7e5b2dc0e5e admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:51.658792 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ff63c3-f5c7-4fd0-a0e1-dbfd8d222529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:51.659462 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ff63c3-f5c7-4fd0-a0e1-dbfd8d222529 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:51.659745 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ff63c3-f5c7-4fd0-a0e1-dbfd8d222529 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:51.660023 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ff63c3-f5c7-4fd0-a0e1-dbfd8d222529 admin admin] PATH_INFO: `/v3/projects/70af716b8d7d4f5583e0370c1a9b4238/users/ce8c2c602fac4ca791762f74973b3508/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:51.661963 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ff63c3-f5c7-4fd0-a0e1-dbfd8d222529 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70af716b8d7d4f5583e0370c1a9b4238, user_id=ce8c2c602fac4ca791762f74973b3508, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:51.664707 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ff63c3-f5c7-4fd0-a0e1-dbfd8d222529 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:51.675935 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 916/1807] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:12:51 2022] PUT /identity/v3/projects/70af716b8d7d4f5583e0370c1a9b4238/users/ce8c2c602fac4ca791762f74973b3508/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:12:51.686681 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1de60d-97c4-4a76-9dae-525ee2603479 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:51.687186 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1de60d-97c4-4a76-9dae-525ee2603479 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:51.687290 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1de60d-97c4-4a76-9dae-525ee2603479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:51.717061 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-8e1de60d-97c4-4a76-9dae-525ee2603479 None None] MFA Rules not processed for user `ce8c2c602fac4ca791762f74973b3508`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:51.718268 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-8e1de60d-97c4-4a76-9dae-525ee2603479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:51.805281 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 892/1808] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:51 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:51.816263 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ff63c3-f5c7-4fd0-a0e1-dbfd8d222529 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:51.818478 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-d9ff63c3-f5c7-4fd0-a0e1-dbfd8d222529 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:51.878807 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ff63c3-f5c7-4fd0-a0e1-dbfd8d222529 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:51.883420 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a06da-388a-46f5-9fb5-e90fa512f005 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:51.884035 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a06da-388a-46f5-9fb5-e90fa512f005 service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:51.884314 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a06da-388a-46f5-9fb5-e90fa512f005 service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:51.884589 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a06da-388a-46f5-9fb5-e90fa512f005 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:51.894326 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81a06da-388a-46f5-9fb5-e90fa512f005 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:51.896960 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81a06da-388a-46f5-9fb5-e90fa512f005 service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:51.907847 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 917/1809] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:12:51 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:12:52.163008 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:52.275457 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:52.906986 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1de60d-97c4-4a76-9dae-525ee2603479 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:52.972542 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1de60d-97c4-4a76-9dae-525ee2603479 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:52.977090 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eb7434-48a4-404e-852c-622b6e3fd35c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:52.977721 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eb7434-48a4-404e-852c-622b6e3fd35c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:52.978007 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eb7434-48a4-404e-852c-622b6e3fd35c admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:52.978282 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eb7434-48a4-404e-852c-622b6e3fd35c admin admin] PATH_INFO: `/v3/users/76ea109f1fc54723a59a8f40d435d4e7` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:52.979424 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eb7434-48a4-404e-852c-622b6e3fd35c admin admin] RBAC: Authorizing `identity:delete_user(user_id=76ea109f1fc54723a59a8f40d435d4e7)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:52.981358 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eb7434-48a4-404e-852c-622b6e3fd35c admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:53.183482 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-37eb7434-48a4-404e-852c-622b6e3fd35c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76ea109f1fc54723a59a8f40d435d4e7'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:53.195775 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-37eb7434-48a4-404e-852c-622b6e3fd35c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76ea109f1fc54723a59a8f40d435d4e7'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:53.196491 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-37eb7434-48a4-404e-852c-622b6e3fd35c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76ea109f1fc54723a59a8f40d435d4e7'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:53.277057 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-37eb7434-48a4-404e-852c-622b6e3fd35c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76ea109f1fc54723a59a8f40d435d4e7'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:53.280212 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-37eb7434-48a4-404e-852c-622b6e3fd35c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76ea109f1fc54723a59a8f40d435d4e7'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:53.309076 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 893/1810] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:12:52 2022] DELETE /identity/v3/users/76ea109f1fc54723a59a8f40d435d4e7 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:12:53.314091 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6853594c-bc06-45f7-90ab-16a0572c605b None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:53.314562 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6853594c-bc06-45f7-90ab-16a0572c605b None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:53.314867 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6853594c-bc06-45f7-90ab-16a0572c605b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:53.344006 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-6853594c-bc06-45f7-90ab-16a0572c605b None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:53.345358 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-6853594c-bc06-45f7-90ab-16a0572c605b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:53.414507 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 918/1811] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:53.424872 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eb7434-48a4-404e-852c-622b6e3fd35c admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:53.426292 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-37eb7434-48a4-404e-852c-622b6e3fd35c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:53.488205 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:53.493337 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eb7434-48a4-404e-852c-622b6e3fd35c admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:53.504174 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeca1b32-889f-4538-a90c-7beab5aa4407 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:53.504824 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeca1b32-889f-4538-a90c-7beab5aa4407 service neutron] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:53.505095 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeca1b32-889f-4538-a90c-7beab5aa4407 service neutron] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:53.505510 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeca1b32-889f-4538-a90c-7beab5aa4407 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:53.523852 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeca1b32-889f-4538-a90c-7beab5aa4407 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:53.525779 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeca1b32-889f-4538-a90c-7beab5aa4407 service neutron] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:53.542950 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 894/1812] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:12:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:12:54.037866 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6853594c-bc06-45f7-90ab-16a0572c605b None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:54.039629 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-6853594c-bc06-45f7-90ab-16a0572c605b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:54.063419 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6853594c-bc06-45f7-90ab-16a0572c605b None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:54.068359 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a605f-1afb-4207-85bc-4de85b5e722e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:54.069496 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a605f-1afb-4207-85bc-4de85b5e722e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:54.070122 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a605f-1afb-4207-85bc-4de85b5e722e admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:54.070417 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a605f-1afb-4207-85bc-4de85b5e722e admin admin] PATH_INFO: `/v3/projects/15a3cdf559004e2986b5415735dff20d` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:54.071703 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023a605f-1afb-4207-85bc-4de85b5e722e admin admin] RBAC: Authorizing `identity:delete_project(project_id=15a3cdf559004e2986b5415735dff20d)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:54.073641 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023a605f-1afb-4207-85bc-4de85b5e722e admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:54.212357 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-023a605f-1afb-4207-85bc-4de85b5e722e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15a3cdf559004e2986b5415735dff20d', 'request_id': 'req-023a605f-1afb-4207-85bc-4de85b5e722e'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:54.219443 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-023a605f-1afb-4207-85bc-4de85b5e722e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15a3cdf559004e2986b5415735dff20d', 'request_id': 'req-023a605f-1afb-4207-85bc-4de85b5e722e'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:54.293913 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-023a605f-1afb-4207-85bc-4de85b5e722e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15a3cdf559004e2986b5415735dff20d', 'request_id': 'req-023a605f-1afb-4207-85bc-4de85b5e722e'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:54.310026 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-023a605f-1afb-4207-85bc-4de85b5e722e admin admin] The token cache is being invalidate because project 15a3cdf559004e2986b5415735dff20d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:12:54.310026 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-023a605f-1afb-4207-85bc-4de85b5e722e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:54.310026 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-023a605f-1afb-4207-85bc-4de85b5e722e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:54.314745 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 919/1813] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:12:54 2022] DELETE /identity/v3/projects/15a3cdf559004e2986b5415735dff20d => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:12:54.315746 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeca1b32-889f-4538-a90c-7beab5aa4407 service neutron] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:54.330588 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeca1b32-889f-4538-a90c-7beab5aa4407 service neutron] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:54.416735 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a081b15-8401-4192-b346-ab1ab413bd67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:54.428882 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a081b15-8401-4192-b346-ab1ab413bd67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:54.428882 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a081b15-8401-4192-b346-ab1ab413bd67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:54.428882 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a081b15-8401-4192-b346-ab1ab413bd67 admin admin] PATH_INFO: `/v3/users/ce8c2c602fac4ca791762f74973b3508` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:54.428882 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a081b15-8401-4192-b346-ab1ab413bd67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce8c2c602fac4ca791762f74973b3508)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:54.428882 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a081b15-8401-4192-b346-ab1ab413bd67 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:55.076744 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-7a081b15-8401-4192-b346-ab1ab413bd67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce8c2c602fac4ca791762f74973b3508'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:55.098950 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-7a081b15-8401-4192-b346-ab1ab413bd67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce8c2c602fac4ca791762f74973b3508'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:55.099994 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-7a081b15-8401-4192-b346-ab1ab413bd67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce8c2c602fac4ca791762f74973b3508'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:55.204994 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-7a081b15-8401-4192-b346-ab1ab413bd67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce8c2c602fac4ca791762f74973b3508'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:55.233875 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-7a081b15-8401-4192-b346-ab1ab413bd67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce8c2c602fac4ca791762f74973b3508'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:55.233875 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 895/1814] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:12:54 2022] DELETE /identity/v3/users/ce8c2c602fac4ca791762f74973b3508 => generated 0 bytes in 906 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:12:55.863391 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a605f-1afb-4207-85bc-4de85b5e722e admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:55.869418 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-023a605f-1afb-4207-85bc-4de85b5e722e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:55.892381 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a605f-1afb-4207-85bc-4de85b5e722e admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:55.915201 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:55.985004 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd686ce2-74ec-4ed8-b054-779b1f895af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:55.985633 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd686ce2-74ec-4ed8-b054-779b1f895af5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:55.985912 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd686ce2-74ec-4ed8-b054-779b1f895af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:55.986186 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd686ce2-74ec-4ed8-b054-779b1f895af5 admin admin] PATH_INFO: `/v3/projects/70af716b8d7d4f5583e0370c1a9b4238` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:55.988337 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd686ce2-74ec-4ed8-b054-779b1f895af5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70af716b8d7d4f5583e0370c1a9b4238)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:55.992492 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd686ce2-74ec-4ed8-b054-779b1f895af5 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:56.148577 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-cd686ce2-74ec-4ed8-b054-779b1f895af5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70af716b8d7d4f5583e0370c1a9b4238', 'request_id': 'req-cd686ce2-74ec-4ed8-b054-779b1f895af5'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:56.156590 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-cd686ce2-74ec-4ed8-b054-779b1f895af5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70af716b8d7d4f5583e0370c1a9b4238', 'request_id': 'req-cd686ce2-74ec-4ed8-b054-779b1f895af5'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:56.218968 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-cd686ce2-74ec-4ed8-b054-779b1f895af5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70af716b8d7d4f5583e0370c1a9b4238', 'request_id': 'req-cd686ce2-74ec-4ed8-b054-779b1f895af5'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:56.239347 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-cd686ce2-74ec-4ed8-b054-779b1f895af5 admin admin] The token cache is being invalidate because project 70af716b8d7d4f5583e0370c1a9b4238 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:12:56.239690 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-cd686ce2-74ec-4ed8-b054-779b1f895af5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:56.240438 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-cd686ce2-74ec-4ed8-b054-779b1f895af5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:56.243288 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 920/1815] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:12:55 2022] DELETE /identity/v3/projects/70af716b8d7d4f5583e0370c1a9b4238 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:12:56.248019 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed684d7-83e8-421a-9955-7fb4b439fb3e None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:56.248901 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed684d7-83e8-421a-9955-7fb4b439fb3e None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:56.249334 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed684d7-83e8-421a-9955-7fb4b439fb3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:56.275830 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-7ed684d7-83e8-421a-9955-7fb4b439fb3e None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:56.277279 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-7ed684d7-83e8-421a-9955-7fb4b439fb3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:56.381806 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 896/1816] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:56.383372 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fca9b8a-ed56-4142-b960-f1547d19426a None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:56.384211 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fca9b8a-ed56-4142-b960-f1547d19426a None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:56.384655 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fca9b8a-ed56-4142-b960-f1547d19426a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:56.437697 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-7fca9b8a-ed56-4142-b960-f1547d19426a None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:56.439367 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-7fca9b8a-ed56-4142-b960-f1547d19426a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:56.448783 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 921/1817] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:56.453355 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed684d7-83e8-421a-9955-7fb4b439fb3e None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:56.463566 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed684d7-83e8-421a-9955-7fb4b439fb3e None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:56.468665 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14312995-fe98-4fa3-ba52-01902ecc7441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:56.469606 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14312995-fe98-4fa3-ba52-01902ecc7441 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:56.470242 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14312995-fe98-4fa3-ba52-01902ecc7441 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:56.470803 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14312995-fe98-4fa3-ba52-01902ecc7441 admin admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:56.471871 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14312995-fe98-4fa3-ba52-01902ecc7441 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:12:56.472390 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14312995-fe98-4fa3-ba52-01902ecc7441 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:56.474276 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14312995-fe98-4fa3-ba52-01902ecc7441 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:56.483780 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:56.494952 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 897/1818] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:12:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 17:12:56.496479 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fca9b8a-ed56-4142-b960-f1547d19426a None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:56.506365 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fca9b8a-ed56-4142-b960-f1547d19426a None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:56.514060 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa055e1-0241-4b22-a929-399f5d581bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:56.514992 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa055e1-0241-4b22-a929-399f5d581bca admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:56.515391 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa055e1-0241-4b22-a929-399f5d581bca admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:56.515701 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa055e1-0241-4b22-a929-399f5d581bca admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:56.516288 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa055e1-0241-4b22-a929-399f5d581bca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:56.518939 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa055e1-0241-4b22-a929-399f5d581bca admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:56.573025 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14312995-fe98-4fa3-ba52-01902ecc7441 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:56.574878 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-14312995-fe98-4fa3-ba52-01902ecc7441 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:56.588538 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14312995-fe98-4fa3-ba52-01902ecc7441 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:56.593698 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1f4724-2874-4869-b00f-38b8976b573c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:56.594358 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f4724-2874-4869-b00f-38b8976b573c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:56.594683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f4724-2874-4869-b00f-38b8976b573c admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:56.595017 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f4724-2874-4869-b00f-38b8976b573c admin admin] PATH_INFO: `/v3/users/b0013b653d7e4a06b1fc0ead54353d81` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:56.596381 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1f4724-2874-4869-b00f-38b8976b573c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0013b653d7e4a06b1fc0ead54353d81)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:56.598463 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1f4724-2874-4869-b00f-38b8976b573c admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:56.639091 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 922/1819] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:12:56 2022] POST /identity/v3/projects => generated 391 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:12:56.646444 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa055e1-0241-4b22-a929-399f5d581bca admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:56.728694 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa055e1-0241-4b22-a929-399f5d581bca admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:56.743048 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187d407b-efb7-4840-a7fe-6fe762b57961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:56.743701 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d407b-efb7-4840-a7fe-6fe762b57961 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:56.743991 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d407b-efb7-4840-a7fe-6fe762b57961 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:56.744271 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d407b-efb7-4840-a7fe-6fe762b57961 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:56.744917 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187d407b-efb7-4840-a7fe-6fe762b57961 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:56.760424 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187d407b-efb7-4840-a7fe-6fe762b57961 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:56.783508 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-5c1f4724-2874-4869-b00f-38b8976b573c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0013b653d7e4a06b1fc0ead54353d81'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:56.799909 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-5c1f4724-2874-4869-b00f-38b8976b573c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0013b653d7e4a06b1fc0ead54353d81'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:56.801184 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-5c1f4724-2874-4869-b00f-38b8976b573c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0013b653d7e4a06b1fc0ead54353d81'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:56.871886 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 923/1820] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:12:56 2022] POST /identity/v3/users => generated 296 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:12:56.880487 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187d407b-efb7-4840-a7fe-6fe762b57961 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:56.908308 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187d407b-efb7-4840-a7fe-6fe762b57961 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:56.918081 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bbdde3-cd5c-44f8-923f-d569d337864f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:56.918723 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bbdde3-cd5c-44f8-923f-d569d337864f admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:56.919200 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bbdde3-cd5c-44f8-923f-d569d337864f admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:56.919360 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bbdde3-cd5c-44f8-923f-d569d337864f admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:56.920002 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bbdde3-cd5c-44f8-923f-d569d337864f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:56.922142 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bbdde3-cd5c-44f8-923f-d569d337864f admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:56.947117 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 924/1821] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:12:56 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 17:12:56.947651 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-5c1f4724-2874-4869-b00f-38b8976b573c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0013b653d7e4a06b1fc0ead54353d81'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:56.948395 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-5c1f4724-2874-4869-b00f-38b8976b573c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0013b653d7e4a06b1fc0ead54353d81'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:56.950158 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bbdde3-cd5c-44f8-923f-d569d337864f admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:56.955314 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-d4bbdde3-cd5c-44f8-923f-d569d337864f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:56.967881 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 898/1822] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:12:56 2022] DELETE /identity/v3/users/b0013b653d7e4a06b1fc0ead54353d81 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:12:56.972453 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55af5089-4333-4b9b-ab5f-c73fd486bcdc None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:56.975715 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55af5089-4333-4b9b-ab5f-c73fd486bcdc None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:56.976444 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55af5089-4333-4b9b-ab5f-c73fd486bcdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:56.988313 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bbdde3-cd5c-44f8-923f-d569d337864f admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:57.032019 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-55af5089-4333-4b9b-ab5f-c73fd486bcdc None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:57.033998 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-55af5089-4333-4b9b-ab5f-c73fd486bcdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:57.155457 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:57.168028 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 899/1823] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:57.168909 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55af5089-4333-4b9b-ab5f-c73fd486bcdc None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:57.171769 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-55af5089-4333-4b9b-ab5f-c73fd486bcdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:57.183804 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0744ae01-40eb-4b21-8667-7c21418082bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:57.184484 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0744ae01-40eb-4b21-8667-7c21418082bb admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:57.184769 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0744ae01-40eb-4b21-8667-7c21418082bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:57.185043 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0744ae01-40eb-4b21-8667-7c21418082bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:57.185748 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0744ae01-40eb-4b21-8667-7c21418082bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:57.187396 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0744ae01-40eb-4b21-8667-7c21418082bb admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:57.204787 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55af5089-4333-4b9b-ab5f-c73fd486bcdc None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:57.211988 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 925/1824] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:12:56 2022] GET /identity/v3/roles => generated 1420 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:12:57.212927 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5944f0-7928-4e9d-b7a8-899bde9c587f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:57.213586 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5944f0-7928-4e9d-b7a8-899bde9c587f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:57.213868 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5944f0-7928-4e9d-b7a8-899bde9c587f admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:57.222013 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0744ae01-40eb-4b21-8667-7c21418082bb admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:57.222431 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5944f0-7928-4e9d-b7a8-899bde9c587f admin admin] PATH_INFO: `/v3/users/f7d69bfc621d4729b13b4381e2e3ed10` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:57.224237 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5944f0-7928-4e9d-b7a8-899bde9c587f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7d69bfc621d4729b13b4381e2e3ed10)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:57.231448 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5944f0-7928-4e9d-b7a8-899bde9c587f admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:57.231892 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-0744ae01-40eb-4b21-8667-7c21418082bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:57.322302 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0744ae01-40eb-4b21-8667-7c21418082bb admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:57.327196 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7b838c-69d8-4027-9230-99e4d4e32e96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:57.327873 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b838c-69d8-4027-9230-99e4d4e32e96 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:57.341194 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b838c-69d8-4027-9230-99e4d4e32e96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:57.341194 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b838c-69d8-4027-9230-99e4d4e32e96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:57.341194 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7b838c-69d8-4027-9230-99e4d4e32e96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:57.341194 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7b838c-69d8-4027-9230-99e4d4e32e96 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:57.353130 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 926/1825] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:12:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:12:57.357664 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7b838c-69d8-4027-9230-99e4d4e32e96 service neutron] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:57.374063 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7b838c-69d8-4027-9230-99e4d4e32e96 service neutron] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:57.379872 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63c9a6c-78f5-47a7-b25f-d6ab503cb937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:57.385096 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c9a6c-78f5-47a7-b25f-d6ab503cb937 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:57.385096 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c9a6c-78f5-47a7-b25f-d6ab503cb937 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:57.385096 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c9a6c-78f5-47a7-b25f-d6ab503cb937 admin admin] PATH_INFO: `/v3/projects/65eaaaeb789f43bca77172f33fa7918a/users/2f1c9be4b5a34d6b89ca27d34e4e425c/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:57.401079 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63c9a6c-78f5-47a7-b25f-d6ab503cb937 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65eaaaeb789f43bca77172f33fa7918a, user_id=2f1c9be4b5a34d6b89ca27d34e4e425c, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:57.409612 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63c9a6c-78f5-47a7-b25f-d6ab503cb937 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:57.591269 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 927/1826] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:12:57 2022] PUT /identity/v3/projects/65eaaaeb789f43bca77172f33fa7918a/users/2f1c9be4b5a34d6b89ca27d34e4e425c/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:12:57.599991 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c098d0b-c0f9-445b-ad6e-427b1a5a2994 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:57.600981 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c098d0b-c0f9-445b-ad6e-427b1a5a2994 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:57.601423 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c098d0b-c0f9-445b-ad6e-427b1a5a2994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:57.629839 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-dc5944f0-7928-4e9d-b7a8-899bde9c587f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7d69bfc621d4729b13b4381e2e3ed10'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:57.632622 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-2c098d0b-c0f9-445b-ad6e-427b1a5a2994 None None] MFA Rules not processed for user `2f1c9be4b5a34d6b89ca27d34e4e425c`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:57.634079 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-2c098d0b-c0f9-445b-ad6e-427b1a5a2994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:57.643628 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-dc5944f0-7928-4e9d-b7a8-899bde9c587f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7d69bfc621d4729b13b4381e2e3ed10'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:57.646422 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-dc5944f0-7928-4e9d-b7a8-899bde9c587f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7d69bfc621d4729b13b4381e2e3ed10'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:57.646422 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:57.692917 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:57.712337 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 928/1827] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:57 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:57.718092 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-dc5944f0-7928-4e9d-b7a8-899bde9c587f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7d69bfc621d4729b13b4381e2e3ed10'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:57.719273 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-dc5944f0-7928-4e9d-b7a8-899bde9c587f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7d69bfc621d4729b13b4381e2e3ed10'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:57.721217 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c098d0b-c0f9-445b-ad6e-427b1a5a2994 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:57.735288 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 900/1828] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:12:57 2022] DELETE /identity/v3/users/f7d69bfc621d4729b13b4381e2e3ed10 => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:12:57.739806 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d244fe-f50a-488d-9b35-179188b9ab38 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:57.740096 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d244fe-f50a-488d-9b35-179188b9ab38 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:57.740386 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d244fe-f50a-488d-9b35-179188b9ab38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:57.761349 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-00d244fe-f50a-488d-9b35-179188b9ab38 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:57.762949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-00d244fe-f50a-488d-9b35-179188b9ab38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:57.807254 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c098d0b-c0f9-445b-ad6e-427b1a5a2994 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:57.830233 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 901/1829] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:57.841482 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d244fe-f50a-488d-9b35-179188b9ab38 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:57.842868 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-00d244fe-f50a-488d-9b35-179188b9ab38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:57.866168 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5860252a-b0c8-40c2-8381-38b987e8e51e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:57.866777 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5860252a-b0c8-40c2-8381-38b987e8e51e admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:57.867096 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5860252a-b0c8-40c2-8381-38b987e8e51e admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:57.867383 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5860252a-b0c8-40c2-8381-38b987e8e51e admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:57.868012 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5860252a-b0c8-40c2-8381-38b987e8e51e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:57.869924 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5860252a-b0c8-40c2-8381-38b987e8e51e admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:57.904244 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d244fe-f50a-488d-9b35-179188b9ab38 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:57.917410 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430c3b3b-7cf7-49f7-a5cf-a4d5d2e9f6bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:57.917410 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430c3b3b-7cf7-49f7-a5cf-a4d5d2e9f6bc service neutron] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:57.917410 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430c3b3b-7cf7-49f7-a5cf-a4d5d2e9f6bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:57.917410 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430c3b3b-7cf7-49f7-a5cf-a4d5d2e9f6bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:57.918475 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430c3b3b-7cf7-49f7-a5cf-a4d5d2e9f6bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:57.919976 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430c3b3b-7cf7-49f7-a5cf-a4d5d2e9f6bc service neutron] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:57.930078 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 902/1830] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:12:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 17:12:57.930957 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430c3b3b-7cf7-49f7-a5cf-a4d5d2e9f6bc service neutron] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:57.932253 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-430c3b3b-7cf7-49f7-a5cf-a4d5d2e9f6bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:57.945612 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430c3b3b-7cf7-49f7-a5cf-a4d5d2e9f6bc service neutron] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:57.950144 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f38283-308f-4465-b9b6-9f1ff1e7ab8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:57.950858 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f38283-308f-4465-b9b6-9f1ff1e7ab8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:57.951185 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f38283-308f-4465-b9b6-9f1ff1e7ab8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:57.951464 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f38283-308f-4465-b9b6-9f1ff1e7ab8f admin admin] PATH_INFO: `/v3/projects/9e46398f6f034daaab53162018c771a4` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:57.952603 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f38283-308f-4465-b9b6-9f1ff1e7ab8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e46398f6f034daaab53162018c771a4)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:57.954700 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f38283-308f-4465-b9b6-9f1ff1e7ab8f admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:57.994409 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 929/1831] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:12:57 2022] POST /identity/v3/projects => generated 393 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 17:12:57.998392 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5860252a-b0c8-40c2-8381-38b987e8e51e admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:58.000142 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-5860252a-b0c8-40c2-8381-38b987e8e51e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:58.068289 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5860252a-b0c8-40c2-8381-38b987e8e51e admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:58.082515 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-09f38283-308f-4465-b9b6-9f1ff1e7ab8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e46398f6f034daaab53162018c771a4', 'request_id': 'req-09f38283-308f-4465-b9b6-9f1ff1e7ab8f'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:58.109464 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:58.142503 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f332817-fef0-483a-b569-cd430e969594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:58.143386 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f332817-fef0-483a-b569-cd430e969594 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:58.145631 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f332817-fef0-483a-b569-cd430e969594 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:58.145631 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f332817-fef0-483a-b569-cd430e969594 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:58.145631 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f332817-fef0-483a-b569-cd430e969594 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:58.146591 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f332817-fef0-483a-b569-cd430e969594 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:58.175408 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-09f38283-308f-4465-b9b6-9f1ff1e7ab8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e46398f6f034daaab53162018c771a4', 'request_id': 'req-09f38283-308f-4465-b9b6-9f1ff1e7ab8f'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:58.183895 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-09f38283-308f-4465-b9b6-9f1ff1e7ab8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e46398f6f034daaab53162018c771a4', 'request_id': 'req-09f38283-308f-4465-b9b6-9f1ff1e7ab8f'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:58.197769 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-09f38283-308f-4465-b9b6-9f1ff1e7ab8f admin admin] The token cache is being invalidate because project 9e46398f6f034daaab53162018c771a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81650) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:12:58.198017 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-09f38283-308f-4465-b9b6-9f1ff1e7ab8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:58.198854 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-09f38283-308f-4465-b9b6-9f1ff1e7ab8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:58.201130 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 903/1832] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:12:57 2022] DELETE /identity/v3/projects/9e46398f6f034daaab53162018c771a4 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:12:58.204940 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f38283-308f-4465-b9b6-9f1ff1e7ab8f admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:58.206443 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-09f38283-308f-4465-b9b6-9f1ff1e7ab8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:58.221565 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 930/1833] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:12:57 2022] POST /identity/v3/users => generated 303 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:12:58.225931 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f332817-fef0-483a-b569-cd430e969594 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:58.227689 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-9f332817-fef0-483a-b569-cd430e969594 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:58.234336 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f38283-308f-4465-b9b6-9f1ff1e7ab8f admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:58.242760 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3f6a76-d05c-4ce0-89dd-24ec6343a956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:58.243470 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f6a76-d05c-4ce0-89dd-24ec6343a956 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:58.243860 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f6a76-d05c-4ce0-89dd-24ec6343a956 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:58.244837 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f6a76-d05c-4ce0-89dd-24ec6343a956 admin admin] PATH_INFO: `/v3/users/db00dca6a8ca4c15bd0ecfc7a0d35e64` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:58.246320 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3f6a76-d05c-4ce0-89dd-24ec6343a956 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db00dca6a8ca4c15bd0ecfc7a0d35e64)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:58.248912 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3f6a76-d05c-4ce0-89dd-24ec6343a956 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:58.255553 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f332817-fef0-483a-b569-cd430e969594 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:58.272273 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a0d221-86a2-4be4-8753-195175575d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:58.273123 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a0d221-86a2-4be4-8753-195175575d96 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:58.273407 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a0d221-86a2-4be4-8753-195175575d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:58.273691 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a0d221-86a2-4be4-8753-195175575d96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:58.274555 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a0d221-86a2-4be4-8753-195175575d96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:58.275959 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a0d221-86a2-4be4-8753-195175575d96 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:58.289973 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:INSERT=2,keystone:DELETE=8,keystone:SELECT=90 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:58.293869 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 931/1834] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:12:58 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 17:12:58.298244 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a0d221-86a2-4be4-8753-195175575d96 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:58.308346 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a0d221-86a2-4be4-8753-195175575d96 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:58.312988 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd018e-814d-4600-a060-f62363c84575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:58.316349 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd018e-814d-4600-a060-f62363c84575 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:58.316349 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd018e-814d-4600-a060-f62363c84575 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:58.316349 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd018e-814d-4600-a060-f62363c84575 admin admin] PATH_INFO: `/v3/projects/bc4d0439de81404581013e253274acc7/users/3b0a67092cef4e9ca9bea3694007703e/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:58.329192 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fd018e-814d-4600-a060-f62363c84575 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc4d0439de81404581013e253274acc7, user_id=3b0a67092cef4e9ca9bea3694007703e, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:58.331885 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fd018e-814d-4600-a060-f62363c84575 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:58.450823 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 932/1835] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:12:58 2022] PUT /identity/v3/projects/bc4d0439de81404581013e253274acc7/users/3b0a67092cef4e9ca9bea3694007703e/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:12:58.451860 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd018e-814d-4600-a060-f62363c84575 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:58.454027 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-02fd018e-814d-4600-a060-f62363c84575 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:58.531502 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd018e-814d-4600-a060-f62363c84575 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:58.536530 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be413900-360a-4d1b-a167-4e064a44f26a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:58.537675 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be413900-360a-4d1b-a167-4e064a44f26a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:58.537996 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be413900-360a-4d1b-a167-4e064a44f26a admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:58.538499 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be413900-360a-4d1b-a167-4e064a44f26a admin admin] PATH_INFO: `/v3/projects/befa67f784fe4ebc8d67a70870d5c37c` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:58.539739 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be413900-360a-4d1b-a167-4e064a44f26a admin admin] RBAC: Authorizing `identity:delete_project(project_id=befa67f784fe4ebc8d67a70870d5c37c)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:58.541678 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be413900-360a-4d1b-a167-4e064a44f26a admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:58.567667 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-3b3f6a76-d05c-4ce0-89dd-24ec6343a956 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db00dca6a8ca4c15bd0ecfc7a0d35e64'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:58.580231 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-3b3f6a76-d05c-4ce0-89dd-24ec6343a956 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db00dca6a8ca4c15bd0ecfc7a0d35e64'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:58.580928 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-3b3f6a76-d05c-4ce0-89dd-24ec6343a956 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db00dca6a8ca4c15bd0ecfc7a0d35e64'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:58.648136 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-3b3f6a76-d05c-4ce0-89dd-24ec6343a956 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db00dca6a8ca4c15bd0ecfc7a0d35e64'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:58.652129 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-3b3f6a76-d05c-4ce0-89dd-24ec6343a956 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db00dca6a8ca4c15bd0ecfc7a0d35e64'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:58.668920 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-be413900-360a-4d1b-a167-4e064a44f26a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'befa67f784fe4ebc8d67a70870d5c37c', 'request_id': 'req-be413900-360a-4d1b-a167-4e064a44f26a'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:58.672929 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 904/1836] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:12:58 2022] DELETE /identity/v3/users/db00dca6a8ca4c15bd0ecfc7a0d35e64 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:12:58.672929 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3f6a76-d05c-4ce0-89dd-24ec6343a956 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:58.672929 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-3b3f6a76-d05c-4ce0-89dd-24ec6343a956 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:58.676513 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-be413900-360a-4d1b-a167-4e064a44f26a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'befa67f784fe4ebc8d67a70870d5c37c', 'request_id': 'req-be413900-360a-4d1b-a167-4e064a44f26a'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:58.746875 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-be413900-360a-4d1b-a167-4e064a44f26a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'befa67f784fe4ebc8d67a70870d5c37c', 'request_id': 'req-be413900-360a-4d1b-a167-4e064a44f26a'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:58.751604 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3f6a76-d05c-4ce0-89dd-24ec6343a956 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:58.763158 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139a978a-bb5b-4880-ab17-c18adb241b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:58.764637 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a978a-bb5b-4880-ab17-c18adb241b26 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:58.765189 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a978a-bb5b-4880-ab17-c18adb241b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:58.765620 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a978a-bb5b-4880-ab17-c18adb241b26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:58.766282 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139a978a-bb5b-4880-ab17-c18adb241b26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:58.767960 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139a978a-bb5b-4880-ab17-c18adb241b26 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:58.771332 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-be413900-360a-4d1b-a167-4e064a44f26a admin admin] The token cache is being invalidate because project befa67f784fe4ebc8d67a70870d5c37c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:12:58.771332 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-be413900-360a-4d1b-a167-4e064a44f26a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:58.773502 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-be413900-360a-4d1b-a167-4e064a44f26a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:58.775426 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 933/1837] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:12:58 2022] DELETE /identity/v3/projects/befa67f784fe4ebc8d67a70870d5c37c => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:12:58.781999 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be413900-360a-4d1b-a167-4e064a44f26a admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:58.784574 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-be413900-360a-4d1b-a167-4e064a44f26a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:58.784675 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 905/1838] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:12:58 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:12:58.793965 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139a978a-bb5b-4880-ab17-c18adb241b26 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:58.800335 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-139a978a-bb5b-4880-ab17-c18adb241b26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:58.822479 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:58.823566 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139a978a-bb5b-4880-ab17-c18adb241b26 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:58.831703 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b618cd5-8a71-4c92-864b-b714e1225ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:58.833291 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b618cd5-8a71-4c92-864b-b714e1225ba4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:58.833291 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b618cd5-8a71-4c92-864b-b714e1225ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:58.833291 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b618cd5-8a71-4c92-864b-b714e1225ba4 admin admin] PATH_INFO: `/v3/projects/bc4d0439de81404581013e253274acc7/users/3b0a67092cef4e9ca9bea3694007703e/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:58.835715 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b618cd5-8a71-4c92-864b-b714e1225ba4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc4d0439de81404581013e253274acc7, user_id=3b0a67092cef4e9ca9bea3694007703e, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:58.842843 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b618cd5-8a71-4c92-864b-b714e1225ba4 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:58.869639 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 906/1839] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:12:58 2022] PUT /identity/v3/projects/bc4d0439de81404581013e253274acc7/users/3b0a67092cef4e9ca9bea3694007703e/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:12:58.880331 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d22439-0428-45cd-a72d-bb6dd4c16ab0 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:58.880628 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d22439-0428-45cd-a72d-bb6dd4c16ab0 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:58.880951 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d22439-0428-45cd-a72d-bb6dd4c16ab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:58.887128 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be413900-360a-4d1b-a167-4e064a44f26a admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:58.953305 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-53d22439-0428-45cd-a72d-bb6dd4c16ab0 None None] MFA Rules not processed for user `3b0a67092cef4e9ca9bea3694007703e`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:58.965573 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-53d22439-0428-45cd-a72d-bb6dd4c16ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:58.999379 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f765e246-a86f-42b2-8eca-eebdfa8f5e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:59.000147 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f765e246-a86f-42b2-8eca-eebdfa8f5e95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:59.000512 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f765e246-a86f-42b2-8eca-eebdfa8f5e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:59.000829 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f765e246-a86f-42b2-8eca-eebdfa8f5e95 admin admin] PATH_INFO: `/v3/users/0a066d988e4648e0847a8bcd406ddfe5` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:59.002076 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f765e246-a86f-42b2-8eca-eebdfa8f5e95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a066d988e4648e0847a8bcd406ddfe5)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:59.004241 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f765e246-a86f-42b2-8eca-eebdfa8f5e95 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:59.034880 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:59.055039 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 907/1840] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:58 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:59.066676 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d22439-0428-45cd-a72d-bb6dd4c16ab0 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:59.068065 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-53d22439-0428-45cd-a72d-bb6dd4c16ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:59.123326 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d22439-0428-45cd-a72d-bb6dd4c16ab0 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:59.127240 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c88fb4-1494-4af5-a2dd-78e8314285d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:59.127850 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c88fb4-1494-4af5-a2dd-78e8314285d2 service cinder] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:59.128130 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c88fb4-1494-4af5-a2dd-78e8314285d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:59.128403 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c88fb4-1494-4af5-a2dd-78e8314285d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:59.137441 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c88fb4-1494-4af5-a2dd-78e8314285d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:59.138985 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c88fb4-1494-4af5-a2dd-78e8314285d2 service cinder] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:59.149077 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 908/1841] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:12:59 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 17:12:59.163695 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c88fb4-1494-4af5-a2dd-78e8314285d2 service cinder] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:59.168147 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-e6c88fb4-1494-4af5-a2dd-78e8314285d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:59.186835 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c88fb4-1494-4af5-a2dd-78e8314285d2 service cinder] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:59.192550 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28449955-ac7c-4105-bdf3-f110d2ee2468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:59.194487 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28449955-ac7c-4105-bdf3-f110d2ee2468 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:59.195139 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28449955-ac7c-4105-bdf3-f110d2ee2468 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:59.195506 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28449955-ac7c-4105-bdf3-f110d2ee2468 admin admin] PATH_INFO: `/v3/projects/0e4973f62a6f4c13a7d66fdcb2fb64c3` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:59.196905 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28449955-ac7c-4105-bdf3-f110d2ee2468 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e4973f62a6f4c13a7d66fdcb2fb64c3)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:59.205110 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28449955-ac7c-4105-bdf3-f110d2ee2468 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:59.224973 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-f765e246-a86f-42b2-8eca-eebdfa8f5e95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a066d988e4648e0847a8bcd406ddfe5'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:59.315754 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-28449955-ac7c-4105-bdf3-f110d2ee2468 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e4973f62a6f4c13a7d66fdcb2fb64c3', 'request_id': 'req-28449955-ac7c-4105-bdf3-f110d2ee2468'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:59.322688 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-f765e246-a86f-42b2-8eca-eebdfa8f5e95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a066d988e4648e0847a8bcd406ddfe5'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:59.323515 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-f765e246-a86f-42b2-8eca-eebdfa8f5e95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a066d988e4648e0847a8bcd406ddfe5'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:59.324826 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-f765e246-a86f-42b2-8eca-eebdfa8f5e95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a066d988e4648e0847a8bcd406ddfe5'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:59.335335 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-f765e246-a86f-42b2-8eca-eebdfa8f5e95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a066d988e4648e0847a8bcd406ddfe5'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:59.354512 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 934/1842] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:12:58 2022] DELETE /identity/v3/users/0a066d988e4648e0847a8bcd406ddfe5 => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:12:59.357820 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f765e246-a86f-42b2-8eca-eebdfa8f5e95 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:59.357820 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-f765e246-a86f-42b2-8eca-eebdfa8f5e95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:59.415730 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:59.432499 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f765e246-a86f-42b2-8eca-eebdfa8f5e95 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:59.437004 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfff5cb-45fe-4eb2-9d61-2b2c876ca981 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:59.437711 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfff5cb-45fe-4eb2-9d61-2b2c876ca981 service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:59.438100 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfff5cb-45fe-4eb2-9d61-2b2c876ca981 service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:59.438450 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfff5cb-45fe-4eb2-9d61-2b2c876ca981 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:59.440432 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-5dfff5cb-45fe-4eb2-9d61-2b2c876ca981 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:59.459775 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-28449955-ac7c-4105-bdf3-f110d2ee2468 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e4973f62a6f4c13a7d66fdcb2fb64c3', 'request_id': 'req-28449955-ac7c-4105-bdf3-f110d2ee2468'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:59.468143 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-28449955-ac7c-4105-bdf3-f110d2ee2468 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e4973f62a6f4c13a7d66fdcb2fb64c3', 'request_id': 'req-28449955-ac7c-4105-bdf3-f110d2ee2468'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:59.483373 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-28449955-ac7c-4105-bdf3-f110d2ee2468 admin admin] The token cache is being invalidate because project 0e4973f62a6f4c13a7d66fdcb2fb64c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81650) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:12:59.483373 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-28449955-ac7c-4105-bdf3-f110d2ee2468 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:59.487387 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-28449955-ac7c-4105-bdf3-f110d2ee2468 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:12:59.491475 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 909/1843] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:12:59 2022] DELETE /identity/v3/projects/0e4973f62a6f4c13a7d66fdcb2fb64c3 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:12:59.494997 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6e0fa7-be6a-450f-84b1-a0593fe9561a None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:59.499868 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6e0fa7-be6a-450f-84b1-a0593fe9561a None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:59.499868 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6e0fa7-be6a-450f-84b1-a0593fe9561a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:59.509678 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfff5cb-45fe-4eb2-9d61-2b2c876ca981 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:59.511136 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfff5cb-45fe-4eb2-9d61-2b2c876ca981 service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:59.514627 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-5f6e0fa7-be6a-450f-84b1-a0593fe9561a None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:59.515866 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-5f6e0fa7-be6a-450f-84b1-a0593fe9561a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:59.520445 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 935/1844] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:12:59 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:12:59.631806 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 910/1845] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:59.642528 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb1c331-6966-4983-b6c1-e779b0f933fb None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:59.642820 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb1c331-6966-4983-b6c1-e779b0f933fb None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:59.643137 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb1c331-6966-4983-b6c1-e779b0f933fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:59.687297 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-2eb1c331-6966-4983-b6c1-e779b0f933fb None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:12:59.688817 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-2eb1c331-6966-4983-b6c1-e779b0f933fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:59.701915 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 936/1846] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:12:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:12:59.703028 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6e0fa7-be6a-450f-84b1-a0593fe9561a None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:59.724484 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6e0fa7-be6a-450f-84b1-a0593fe9561a None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:59.732490 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142f79e8-9cf1-4fe9-a021-a0a4f2e7ff15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:59.738155 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f79e8-9cf1-4fe9-a021-a0a4f2e7ff15 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:59.738155 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f79e8-9cf1-4fe9-a021-a0a4f2e7ff15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:59.738155 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f79e8-9cf1-4fe9-a021-a0a4f2e7ff15 admin admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:59.738155 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142f79e8-9cf1-4fe9-a021-a0a4f2e7ff15 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:12:59.738155 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142f79e8-9cf1-4fe9-a021-a0a4f2e7ff15 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:59.738155 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142f79e8-9cf1-4fe9-a021-a0a4f2e7ff15 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:59.762960 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 911/1847] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:12:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 17:12:59.766277 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb1c331-6966-4983-b6c1-e779b0f933fb None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:59.788343 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb1c331-6966-4983-b6c1-e779b0f933fb None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:59.795309 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f676a7b-7fd9-4942-8166-3a55ab1d8fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:59.795309 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f676a7b-7fd9-4942-8166-3a55ab1d8fe1 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:59.795309 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f676a7b-7fd9-4942-8166-3a55ab1d8fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:59.795309 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f676a7b-7fd9-4942-8166-3a55ab1d8fe1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:59.795627 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f676a7b-7fd9-4942-8166-3a55ab1d8fe1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:59.798304 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f676a7b-7fd9-4942-8166-3a55ab1d8fe1 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:12:59.876077 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142f79e8-9cf1-4fe9-a021-a0a4f2e7ff15 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:12:59.876077 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-142f79e8-9cf1-4fe9-a021-a0a4f2e7ff15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:12:59.876077 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:12:59.883778 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142f79e8-9cf1-4fe9-a021-a0a4f2e7ff15 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:12:59.890138 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb952b0c-eabd-4f25-a50f-04a031511e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:12:59.890919 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb952b0c-eabd-4f25-a50f-04a031511e93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:12:59.891310 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb952b0c-eabd-4f25-a50f-04a031511e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:12:59.891941 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb952b0c-eabd-4f25-a50f-04a031511e93 admin admin] PATH_INFO: `/v3/projects/3ad3ae716b75468cae4b02494944e0e2` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:12:59.893752 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb952b0c-eabd-4f25-a50f-04a031511e93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ad3ae716b75468cae4b02494944e0e2)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:12:59.903696 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb952b0c-eabd-4f25-a50f-04a031511e93 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:00.055963 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 937/1848] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:12:59 2022] POST /identity/v3/projects => generated 371 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:13:00.062783 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f676a7b-7fd9-4942-8166-3a55ab1d8fe1 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:00.138080 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f676a7b-7fd9-4942-8166-3a55ab1d8fe1 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:00.142972 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b0a0ec-6f5c-46ce-b348-5399eeb7fa96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:00.143696 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b0a0ec-6f5c-46ce-b348-5399eeb7fa96 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:00.144045 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b0a0ec-6f5c-46ce-b348-5399eeb7fa96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:00.144377 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b0a0ec-6f5c-46ce-b348-5399eeb7fa96 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:00.145156 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b0a0ec-6f5c-46ce-b348-5399eeb7fa96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:00.147544 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b0a0ec-6f5c-46ce-b348-5399eeb7fa96 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:00.433436 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-eb952b0c-eabd-4f25-a50f-04a031511e93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ad3ae716b75468cae4b02494944e0e2', 'request_id': 'req-eb952b0c-eabd-4f25-a50f-04a031511e93'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:00.449815 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 938/1849] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:13:00 2022] POST /identity/v3/users => generated 286 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:13:00.454078 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b0a0ec-6f5c-46ce-b348-5399eeb7fa96 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:00.464800 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b0a0ec-6f5c-46ce-b348-5399eeb7fa96 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:00.495589 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-eb952b0c-eabd-4f25-a50f-04a031511e93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ad3ae716b75468cae4b02494944e0e2', 'request_id': 'req-eb952b0c-eabd-4f25-a50f-04a031511e93'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:00.501859 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-eb952b0c-eabd-4f25-a50f-04a031511e93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ad3ae716b75468cae4b02494944e0e2', 'request_id': 'req-eb952b0c-eabd-4f25-a50f-04a031511e93'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:00.514669 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-eb952b0c-eabd-4f25-a50f-04a031511e93 admin admin] The token cache is being invalidate because project 3ad3ae716b75468cae4b02494944e0e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81650) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:13:00.514996 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-eb952b0c-eabd-4f25-a50f-04a031511e93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:00.515661 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-eb952b0c-eabd-4f25-a50f-04a031511e93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:00.517170 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 912/1850] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:12:59 2022] DELETE /identity/v3/projects/3ad3ae716b75468cae4b02494944e0e2 => generated 0 bytes in 663 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:13:00.520172 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:13:00.523670 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc39495-b18a-46aa-a03d-54f11608d121 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:00.524087 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc39495-b18a-46aa-a03d-54f11608d121 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:00.524543 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc39495-b18a-46aa-a03d-54f11608d121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:00.527896 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7233aac6-bc37-49d5-a044-4d216a129420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:00.528540 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7233aac6-bc37-49d5-a044-4d216a129420 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:00.528819 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7233aac6-bc37-49d5-a044-4d216a129420 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:00.529090 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7233aac6-bc37-49d5-a044-4d216a129420 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:00.530099 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7233aac6-bc37-49d5-a044-4d216a129420 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:00.531513 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7233aac6-bc37-49d5-a044-4d216a129420 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:00.544252 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 939/1851] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:13:00 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 17:13:00.548250 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7233aac6-bc37-49d5-a044-4d216a129420 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:00.557980 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-5dc39495-b18a-46aa-a03d-54f11608d121 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:13:00.558266 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-7233aac6-bc37-49d5-a044-4d216a129420 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:00.558374 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-5dc39495-b18a-46aa-a03d-54f11608d121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:00.559530 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 913/1852] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:13:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:13:00.564076 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7233aac6-bc37-49d5-a044-4d216a129420 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:00.569019 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471659e7-b7e5-44f2-867a-039ad198dd86 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:00.569450 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471659e7-b7e5-44f2-867a-039ad198dd86 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:00.569578 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf53b2c-9207-4635-b764-1c43cc9e93e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:00.569822 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471659e7-b7e5-44f2-867a-039ad198dd86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:00.570283 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf53b2c-9207-4635-b764-1c43cc9e93e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:00.570635 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf53b2c-9207-4635-b764-1c43cc9e93e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:00.571005 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf53b2c-9207-4635-b764-1c43cc9e93e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:00.571716 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf53b2c-9207-4635-b764-1c43cc9e93e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:00.573382 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf53b2c-9207-4635-b764-1c43cc9e93e0 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:00.586479 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 940/1853] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:13:00 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:13:00.590870 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf53b2c-9207-4635-b764-1c43cc9e93e0 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:00.594820 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-471659e7-b7e5-44f2-867a-039ad198dd86 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:13:00.596159 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-471659e7-b7e5-44f2-867a-039ad198dd86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:00.603358 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf53b2c-9207-4635-b764-1c43cc9e93e0 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:00.608526 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea4054-c61d-40e4-95bd-63a819913532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:00.608697 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 914/1854] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:13:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 17:13:00.609331 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea4054-c61d-40e4-95bd-63a819913532 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:00.609576 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471659e7-b7e5-44f2-867a-039ad198dd86 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:00.609749 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea4054-c61d-40e4-95bd-63a819913532 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:00.610027 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea4054-c61d-40e4-95bd-63a819913532 admin admin] PATH_INFO: `/v3/projects/fbb3035802a84adeaf6a8b2d2203f0ba/users/7dff72da2f78449bbfe895ed38c781fd/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:00.635261 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471659e7-b7e5-44f2-867a-039ad198dd86 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:00.638650 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ea4054-c61d-40e4-95bd-63a819913532 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbb3035802a84adeaf6a8b2d2203f0ba, user_id=7dff72da2f78449bbfe895ed38c781fd, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:00.641752 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ea4054-c61d-40e4-95bd-63a819913532 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:00.651790 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a81deac-d2ff-4592-bf5c-722152f9bb0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:00.652419 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a81deac-d2ff-4592-bf5c-722152f9bb0d admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:00.652700 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a81deac-d2ff-4592-bf5c-722152f9bb0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:00.652992 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a81deac-d2ff-4592-bf5c-722152f9bb0d admin admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:00.653669 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a81deac-d2ff-4592-bf5c-722152f9bb0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:13:00.653930 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a81deac-d2ff-4592-bf5c-722152f9bb0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:00.655709 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a81deac-d2ff-4592-bf5c-722152f9bb0d admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:00.681437 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 915/1855] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:13:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 17:13:00.686385 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a81deac-d2ff-4592-bf5c-722152f9bb0d admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:00.699892 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a81deac-d2ff-4592-bf5c-722152f9bb0d admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:00.707175 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce1b25e-2e75-4acc-a0fa-8ac7fdbeddc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:00.707923 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce1b25e-2e75-4acc-a0fa-8ac7fdbeddc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:00.708302 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce1b25e-2e75-4acc-a0fa-8ac7fdbeddc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:00.708682 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce1b25e-2e75-4acc-a0fa-8ac7fdbeddc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:00.710030 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce1b25e-2e75-4acc-a0fa-8ac7fdbeddc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:00.712829 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce1b25e-2e75-4acc-a0fa-8ac7fdbeddc0 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:00.779343 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 941/1856] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:13:00 2022] PUT /identity/v3/projects/fbb3035802a84adeaf6a8b2d2203f0ba/users/7dff72da2f78449bbfe895ed38c781fd/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:13:00.789903 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d9bf3-6a40-4c0a-98b1-4f24ecfe8fe9 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:00.790194 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d9bf3-6a40-4c0a-98b1-4f24ecfe8fe9 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:00.790471 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d9bf3-6a40-4c0a-98b1-4f24ecfe8fe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:00.821531 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-e53d9bf3-6a40-4c0a-98b1-4f24ecfe8fe9 None None] MFA Rules not processed for user `7dff72da2f78449bbfe895ed38c781fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:13:00.822823 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-e53d9bf3-6a40-4c0a-98b1-4f24ecfe8fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:00.842317 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 916/1857] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:13:00 2022] POST /identity/v3/projects => generated 365 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 17:13:00.843391 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce1b25e-2e75-4acc-a0fa-8ac7fdbeddc0 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:00.961923 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce1b25e-2e75-4acc-a0fa-8ac7fdbeddc0 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:00.972945 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132303e-70fa-4b09-9d74-1d8ea0d5792e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:00.973732 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132303e-70fa-4b09-9d74-1d8ea0d5792e admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:00.974145 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132303e-70fa-4b09-9d74-1d8ea0d5792e admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:00.974792 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132303e-70fa-4b09-9d74-1d8ea0d5792e admin admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:00.975613 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b132303e-70fa-4b09-9d74-1d8ea0d5792e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:00.978193 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b132303e-70fa-4b09-9d74-1d8ea0d5792e admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:00.987722 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 942/1858] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:13:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 17:13:00.993624 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53d9bf3-6a40-4c0a-98b1-4f24ecfe8fe9 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:00.995349 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-e53d9bf3-6a40-4c0a-98b1-4f24ecfe8fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:01.080738 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53d9bf3-6a40-4c0a-98b1-4f24ecfe8fe9 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:01.113017 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 917/1859] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:13:00 2022] POST /identity/v3/users => generated 283 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:13:01.118320 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132303e-70fa-4b09-9d74-1d8ea0d5792e admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:01.134152 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132303e-70fa-4b09-9d74-1d8ea0d5792e admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:01.145758 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b12b7bd-a0bd-4d8d-9f5c-a5d0fa32283b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:01.146428 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b12b7bd-a0bd-4d8d-9f5c-a5d0fa32283b admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:01.146707 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b12b7bd-a0bd-4d8d-9f5c-a5d0fa32283b admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:01.147016 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b12b7bd-a0bd-4d8d-9f5c-a5d0fa32283b admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:01.147670 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b12b7bd-a0bd-4d8d-9f5c-a5d0fa32283b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:01.151676 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b12b7bd-a0bd-4d8d-9f5c-a5d0fa32283b admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:01.152842 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f6f6da-a2a7-463f-840f-959ed706a27a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:01.153965 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f6f6da-a2a7-463f-840f-959ed706a27a service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:01.154352 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f6f6da-a2a7-463f-840f-959ed706a27a service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:01.154824 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f6f6da-a2a7-463f-840f-959ed706a27a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:01.169771 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 918/1860] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:13:01 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 17:13:01.171293 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b12b7bd-a0bd-4d8d-9f5c-a5d0fa32283b admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:01.183657 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b12b7bd-a0bd-4d8d-9f5c-a5d0fa32283b admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:01.184922 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:13:01.191035 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332e74a6-958f-4ebe-b73b-3f589f856dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:01.194250 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e74a6-958f-4ebe-b73b-3f589f856dbe admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:01.194250 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e74a6-958f-4ebe-b73b-3f589f856dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:01.194250 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e74a6-958f-4ebe-b73b-3f589f856dbe admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:01.194250 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332e74a6-958f-4ebe-b73b-3f589f856dbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:01.195256 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332e74a6-958f-4ebe-b73b-3f589f856dbe admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:01.220066 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 919/1861] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:13:01 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:13:01.224413 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332e74a6-958f-4ebe-b73b-3f589f856dbe admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:01.231717 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:13:01.236216 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332e74a6-958f-4ebe-b73b-3f589f856dbe admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:01.243645 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6cbe15-4c1e-4593-a100-5f26c230f402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:01.244419 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6cbe15-4c1e-4593-a100-5f26c230f402 admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:01.244716 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6cbe15-4c1e-4593-a100-5f26c230f402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:01.244997 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6cbe15-4c1e-4593-a100-5f26c230f402 admin admin] PATH_INFO: `/v3/projects/a558e5dae59c4cd2b0dcd5d0990afb18/users/b2e9c8c3fae24896a8b59abedb2e45a9/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:01.245680 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f6f6da-a2a7-463f-840f-959ed706a27a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:01.247376 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f6f6da-a2a7-463f-840f-959ed706a27a service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:01.259261 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 943/1862] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:13:00 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 17:13:01.264073 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6cbe15-4c1e-4593-a100-5f26c230f402 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a558e5dae59c4cd2b0dcd5d0990afb18, user_id=b2e9c8c3fae24896a8b59abedb2e45a9, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:01.266823 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6cbe15-4c1e-4593-a100-5f26c230f402 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:01.281916 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f6f6da-a2a7-463f-840f-959ed706a27a service cinder] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:01.296999 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f6f6da-a2a7-463f-840f-959ed706a27a service cinder] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:01.302242 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5892b3-83cb-40b8-aa2b-664fdd120f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:01.303713 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5892b3-83cb-40b8-aa2b-664fdd120f2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:01.304063 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5892b3-83cb-40b8-aa2b-664fdd120f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:01.304409 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5892b3-83cb-40b8-aa2b-664fdd120f2d admin admin] PATH_INFO: `/v3/users/7dff72da2f78449bbfe895ed38c781fd` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:01.305688 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5892b3-83cb-40b8-aa2b-664fdd120f2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7dff72da2f78449bbfe895ed38c781fd)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:01.307748 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5892b3-83cb-40b8-aa2b-664fdd120f2d admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:01.360096 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 920/1863] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:13:01 2022] PUT /identity/v3/projects/a558e5dae59c4cd2b0dcd5d0990afb18/users/b2e9c8c3fae24896a8b59abedb2e45a9/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:13:01.371859 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b883706-efc2-4acc-932d-c408c47d27f8 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:01.372946 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b883706-efc2-4acc-932d-c408c47d27f8 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:01.373618 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b883706-efc2-4acc-932d-c408c47d27f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:01.423026 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-5b883706-efc2-4acc-932d-c408c47d27f8 None None] MFA Rules not processed for user `b2e9c8c3fae24896a8b59abedb2e45a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:13:01.424291 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-5b883706-efc2-4acc-932d-c408c47d27f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:01.490346 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 921/1864] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:13:01 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:13:01.494797 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b883706-efc2-4acc-932d-c408c47d27f8 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:01.555171 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b883706-efc2-4acc-932d-c408c47d27f8 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:01.559800 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ff5892b3-83cb-40b8-aa2b-664fdd120f2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7dff72da2f78449bbfe895ed38c781fd'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:01.560006 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c491188f-2011-4596-bad3-a43df8f4081a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:01.560615 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c491188f-2011-4596-bad3-a43df8f4081a admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:01.560894 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c491188f-2011-4596-bad3-a43df8f4081a admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:01.561167 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c491188f-2011-4596-bad3-a43df8f4081a admin admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:01.561781 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c491188f-2011-4596-bad3-a43df8f4081a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:01.563919 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c491188f-2011-4596-bad3-a43df8f4081a admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:01.619068 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ff5892b3-83cb-40b8-aa2b-664fdd120f2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7dff72da2f78449bbfe895ed38c781fd'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:01.619619 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ff5892b3-83cb-40b8-aa2b-664fdd120f2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7dff72da2f78449bbfe895ed38c781fd'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:01.620222 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ff5892b3-83cb-40b8-aa2b-664fdd120f2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7dff72da2f78449bbfe895ed38c781fd'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:01.627483 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ff5892b3-83cb-40b8-aa2b-664fdd120f2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7dff72da2f78449bbfe895ed38c781fd'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:01.645010 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 944/1865] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:13:01 2022] DELETE /identity/v3/users/7dff72da2f78449bbfe895ed38c781fd => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:13:01.650365 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 922/1866] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:13:01 2022] POST /identity/v3/projects => generated 365 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 17:13:01.650493 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ccc6f-45b1-4e7f-993f-42a428df0921 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:01.650742 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ccc6f-45b1-4e7f-993f-42a428df0921 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:01.651108 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ccc6f-45b1-4e7f-993f-42a428df0921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:01.654286 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c491188f-2011-4596-bad3-a43df8f4081a admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:01.655886 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-c491188f-2011-4596-bad3-a43df8f4081a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:01.669776 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-e44ccc6f-45b1-4e7f-993f-42a428df0921 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:13:01.671252 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-e44ccc6f-45b1-4e7f-993f-42a428df0921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:01.717763 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:13:01.721512 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c491188f-2011-4596-bad3-a43df8f4081a admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:01.726449 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037bb29f-8335-4138-8775-b4c63cd78dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:01.727079 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037bb29f-8335-4138-8775-b4c63cd78dd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:01.730298 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037bb29f-8335-4138-8775-b4c63cd78dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:01.730298 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037bb29f-8335-4138-8775-b4c63cd78dd7 admin admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:01.730298 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037bb29f-8335-4138-8775-b4c63cd78dd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:01.730298 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037bb29f-8335-4138-8775-b4c63cd78dd7 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:01.733588 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 945/1867] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:13:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 17:13:01.743116 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44ccc6f-45b1-4e7f-993f-42a428df0921 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:01.744631 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-e44ccc6f-45b1-4e7f-993f-42a428df0921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:01.788625 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:13:01.818515 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44ccc6f-45b1-4e7f-993f-42a428df0921 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:01.823404 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af792f5-70cb-4c30-a68c-bff362576c6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:01.824092 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af792f5-70cb-4c30-a68c-bff362576c6a service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:01.824955 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af792f5-70cb-4c30-a68c-bff362576c6a service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:01.825089 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af792f5-70cb-4c30-a68c-bff362576c6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:01.837135 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af792f5-70cb-4c30-a68c-bff362576c6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:01.838594 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af792f5-70cb-4c30-a68c-bff362576c6a service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:01.850245 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 923/1868] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:13:01 2022] POST /identity/v3/users => generated 289 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:13:01.856117 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037bb29f-8335-4138-8775-b4c63cd78dd7 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:01.873911 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037bb29f-8335-4138-8775-b4c63cd78dd7 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:01.880222 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d030fab-3221-4283-9956-a1f5a238ff6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:01.881283 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 946/1869] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:13:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 17:13:01.882149 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d030fab-3221-4283-9956-a1f5a238ff6d admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:01.882442 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d030fab-3221-4283-9956-a1f5a238ff6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:01.882784 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d030fab-3221-4283-9956-a1f5a238ff6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:01.883647 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d030fab-3221-4283-9956-a1f5a238ff6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:01.885776 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d030fab-3221-4283-9956-a1f5a238ff6d admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:01.902967 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 924/1870] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:13:01 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 17:13:01.904156 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af792f5-70cb-4c30-a68c-bff362576c6a service neutron] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:01.915695 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af792f5-70cb-4c30-a68c-bff362576c6a service neutron] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:01.922286 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c825fd4d-ccab-4e64-85e2-96748c1e0ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:01.923233 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825fd4d-ccab-4e64-85e2-96748c1e0ffb admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:01.923681 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825fd4d-ccab-4e64-85e2-96748c1e0ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:01.924124 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825fd4d-ccab-4e64-85e2-96748c1e0ffb admin admin] PATH_INFO: `/v3/projects/274617e664c04b7f8d72d74c09d14593/users/01246c1b553047a69dfbbd4207728451/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:01.952651 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c825fd4d-ccab-4e64-85e2-96748c1e0ffb admin admin] RBAC: Authorizing `identity:create_grant(project_id=274617e664c04b7f8d72d74c09d14593, user_id=01246c1b553047a69dfbbd4207728451, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:01.955644 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c825fd4d-ccab-4e64-85e2-96748c1e0ffb admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:02.061735 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 947/1871] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:13:01 2022] PUT /identity/v3/projects/274617e664c04b7f8d72d74c09d14593/users/01246c1b553047a69dfbbd4207728451/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:13:02.077273 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d030fab-3221-4283-9956-a1f5a238ff6d admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:02.105106 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d030fab-3221-4283-9956-a1f5a238ff6d admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:02.234237 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be3e892-b967-4934-8ae3-618231ea9c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:02.235023 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be3e892-b967-4934-8ae3-618231ea9c3e admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:02.235727 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be3e892-b967-4934-8ae3-618231ea9c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:02.236009 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be3e892-b967-4934-8ae3-618231ea9c3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:02.236653 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be3e892-b967-4934-8ae3-618231ea9c3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:02.238338 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be3e892-b967-4934-8ae3-618231ea9c3e admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:02.251404 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 925/1872] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:13:02 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:13:02.256284 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c825fd4d-ccab-4e64-85e2-96748c1e0ffb admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:02.268069 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c825fd4d-ccab-4e64-85e2-96748c1e0ffb admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:02.280275 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cba53da-9eac-4248-9053-1a7540272035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:02.280956 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba53da-9eac-4248-9053-1a7540272035 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:02.281598 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba53da-9eac-4248-9053-1a7540272035 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:02.281598 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba53da-9eac-4248-9053-1a7540272035 admin admin] PATH_INFO: `/v3/projects/274617e664c04b7f8d72d74c09d14593/users/01246c1b553047a69dfbbd4207728451/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:02.283661 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cba53da-9eac-4248-9053-1a7540272035 admin admin] RBAC: Authorizing `identity:create_grant(project_id=274617e664c04b7f8d72d74c09d14593, user_id=01246c1b553047a69dfbbd4207728451, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:02.290767 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cba53da-9eac-4248-9053-1a7540272035 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:02.309508 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 948/1873] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:13:02 2022] PUT /identity/v3/projects/274617e664c04b7f8d72d74c09d14593/users/01246c1b553047a69dfbbd4207728451/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:13:02.320611 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5712db-d06e-465c-bb3a-c3d9f65c527a None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:02.320895 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5712db-d06e-465c-bb3a-c3d9f65c527a None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:02.324272 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5712db-d06e-465c-bb3a-c3d9f65c527a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:02.350690 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-ca5712db-d06e-465c-bb3a-c3d9f65c527a None None] MFA Rules not processed for user `01246c1b553047a69dfbbd4207728451`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:13:02.354863 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-ca5712db-d06e-465c-bb3a-c3d9f65c527a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:02.419938 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cba53da-9eac-4248-9053-1a7540272035 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:02.422744 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-3cba53da-9eac-4248-9053-1a7540272035 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:02.435638 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 926/1874] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:13:02 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:13:02.452149 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5712db-d06e-465c-bb3a-c3d9f65c527a None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:02.454721 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-ca5712db-d06e-465c-bb3a-c3d9f65c527a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:02.516706 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cba53da-9eac-4248-9053-1a7540272035 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:02.522031 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424e049a-8a4d-4252-9f5b-b81bed16a519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:02.522661 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424e049a-8a4d-4252-9f5b-b81bed16a519 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:02.522987 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424e049a-8a4d-4252-9f5b-b81bed16a519 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:02.523265 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424e049a-8a4d-4252-9f5b-b81bed16a519 admin admin] PATH_INFO: `/v3/projects/fbb3035802a84adeaf6a8b2d2203f0ba` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:02.524377 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424e049a-8a4d-4252-9f5b-b81bed16a519 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbb3035802a84adeaf6a8b2d2203f0ba)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:02.526643 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424e049a-8a4d-4252-9f5b-b81bed16a519 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:02.530978 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5712db-d06e-465c-bb3a-c3d9f65c527a None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:02.534683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6634f4c-2f9f-4179-8241-e9fa0b40cd1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:02.535329 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6634f4c-2f9f-4179-8241-e9fa0b40cd1d service cinder] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:02.535612 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6634f4c-2f9f-4179-8241-e9fa0b40cd1d service cinder] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:02.535887 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6634f4c-2f9f-4179-8241-e9fa0b40cd1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:02.545082 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6634f4c-2f9f-4179-8241-e9fa0b40cd1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:02.546788 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6634f4c-2f9f-4179-8241-e9fa0b40cd1d service cinder] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:02.550677 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:13:02.557368 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 927/1875] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:13:02 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:13:02.627046 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-424e049a-8a4d-4252-9f5b-b81bed16a519 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbb3035802a84adeaf6a8b2d2203f0ba', 'request_id': 'req-424e049a-8a4d-4252-9f5b-b81bed16a519'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:02.634084 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-424e049a-8a4d-4252-9f5b-b81bed16a519 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbb3035802a84adeaf6a8b2d2203f0ba', 'request_id': 'req-424e049a-8a4d-4252-9f5b-b81bed16a519'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:02.738875 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-424e049a-8a4d-4252-9f5b-b81bed16a519 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbb3035802a84adeaf6a8b2d2203f0ba', 'request_id': 'req-424e049a-8a4d-4252-9f5b-b81bed16a519'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:02.751373 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-424e049a-8a4d-4252-9f5b-b81bed16a519 admin admin] The token cache is being invalidate because project fbb3035802a84adeaf6a8b2d2203f0ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:13:02.751670 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-424e049a-8a4d-4252-9f5b-b81bed16a519 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:02.752340 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-424e049a-8a4d-4252-9f5b-b81bed16a519 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:02.753849 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 949/1876] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:13:02 2022] DELETE /identity/v3/projects/fbb3035802a84adeaf6a8b2d2203f0ba => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:13:02.759898 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6634f4c-2f9f-4179-8241-e9fa0b40cd1d service cinder] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:02.760463 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc031ce9-dd92-40d0-9684-3ca39a594dc8 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:02.760750 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc031ce9-dd92-40d0-9684-3ca39a594dc8 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:02.761026 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc031ce9-dd92-40d0-9684-3ca39a594dc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:02.770840 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6634f4c-2f9f-4179-8241-e9fa0b40cd1d service cinder] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:02.787406 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-fc031ce9-dd92-40d0-9684-3ca39a594dc8 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:13:02.788912 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-fc031ce9-dd92-40d0-9684-3ca39a594dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:02.813947 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:13:02.845343 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f728c3b0-3d15-4a8a-a076-7ed9c41a6cf5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:02.846353 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f728c3b0-3d15-4a8a-a076-7ed9c41a6cf5 service cinder] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:02.846995 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f728c3b0-3d15-4a8a-a076-7ed9c41a6cf5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:02.847455 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f728c3b0-3d15-4a8a-a076-7ed9c41a6cf5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:02.849886 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-f728c3b0-3d15-4a8a-a076-7ed9c41a6cf5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:02.863345 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 950/1877] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:13:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 17:13:02.870839 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3574e6e-8b13-459e-8feb-cdad39dfc1d8 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:02.875956 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3574e6e-8b13-459e-8feb-cdad39dfc1d8 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:02.875956 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3574e6e-8b13-459e-8feb-cdad39dfc1d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:02.901795 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-c3574e6e-8b13-459e-8feb-cdad39dfc1d8 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:13:02.903542 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-c3574e6e-8b13-459e-8feb-cdad39dfc1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:02.913825 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 951/1878] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:13:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:13:02.917850 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3574e6e-8b13-459e-8feb-cdad39dfc1d8 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:02.927315 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3574e6e-8b13-459e-8feb-cdad39dfc1d8 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:02.933025 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f851a0b1-9ff9-4c69-96e7-124779b6152e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:02.933668 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f851a0b1-9ff9-4c69-96e7-124779b6152e admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:02.933946 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f851a0b1-9ff9-4c69-96e7-124779b6152e admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:02.934213 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f851a0b1-9ff9-4c69-96e7-124779b6152e admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:02.934851 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f851a0b1-9ff9-4c69-96e7-124779b6152e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:13:02.935205 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f851a0b1-9ff9-4c69-96e7-124779b6152e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:02.936805 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f851a0b1-9ff9-4c69-96e7-124779b6152e admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:02.939313 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f728c3b0-3d15-4a8a-a076-7ed9c41a6cf5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:02.940898 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f728c3b0-3d15-4a8a-a076-7ed9c41a6cf5 service cinder] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:02.949924 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 952/1879] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:13:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 17:13:02.950669 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 928/1880] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:13:02 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 17:13:02.954074 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f851a0b1-9ff9-4c69-96e7-124779b6152e admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:02.965702 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f851a0b1-9ff9-4c69-96e7-124779b6152e admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:02.972291 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5577a56-f2e3-4e15-b191-9837e3ef41b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:02.973397 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5577a56-f2e3-4e15-b191-9837e3ef41b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:02.973854 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5577a56-f2e3-4e15-b191-9837e3ef41b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:02.974258 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5577a56-f2e3-4e15-b191-9837e3ef41b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:02.975686 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5577a56-f2e3-4e15-b191-9837e3ef41b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:02.978190 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5577a56-f2e3-4e15-b191-9837e3ef41b1 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:03.108292 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 953/1881] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:13:02 2022] POST /identity/v3/projects => generated 375 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 17:13:03.112586 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f728c3b0-3d15-4a8a-a076-7ed9c41a6cf5 service cinder] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:03.126203 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f728c3b0-3d15-4a8a-a076-7ed9c41a6cf5 service cinder] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:03.187439 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4a3d00-d394-4144-b38a-4df0d96ed83a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:03.188615 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a3d00-d394-4144-b38a-4df0d96ed83a admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:03.188981 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a3d00-d394-4144-b38a-4df0d96ed83a admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:03.189366 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a3d00-d394-4144-b38a-4df0d96ed83a admin admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:03.190050 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4a3d00-d394-4144-b38a-4df0d96ed83a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:03.192109 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4a3d00-d394-4144-b38a-4df0d96ed83a admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:03.270349 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 929/1882] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:13:03 2022] POST /identity/v3/users => generated 288 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:13:03.275711 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5577a56-f2e3-4e15-b191-9837e3ef41b1 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:03.286389 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5577a56-f2e3-4e15-b191-9837e3ef41b1 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:03.291038 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b357d1-f4f6-4f72-99ea-b9321995dd6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:03.291991 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b357d1-f4f6-4f72-99ea-b9321995dd6f admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:03.302242 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b357d1-f4f6-4f72-99ea-b9321995dd6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:03.302242 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b357d1-f4f6-4f72-99ea-b9321995dd6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:03.302242 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b357d1-f4f6-4f72-99ea-b9321995dd6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:03.302242 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b357d1-f4f6-4f72-99ea-b9321995dd6f admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:03.315148 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 954/1883] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:13:03 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:13:03.316370 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4a3d00-d394-4144-b38a-4df0d96ed83a admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:03.327526 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4a3d00-d394-4144-b38a-4df0d96ed83a admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:03.332262 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8aa7da-0fa5-4203-b33a-4155656e330b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:03.332863 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8aa7da-0fa5-4203-b33a-4155656e330b admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:03.333142 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8aa7da-0fa5-4203-b33a-4155656e330b admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:03.333412 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8aa7da-0fa5-4203-b33a-4155656e330b admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:03.334274 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8aa7da-0fa5-4203-b33a-4155656e330b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:03.336041 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8aa7da-0fa5-4203-b33a-4155656e330b admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:03.346791 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 930/1884] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:13:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:13:03.351510 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b357d1-f4f6-4f72-99ea-b9321995dd6f admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:03.361597 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b357d1-f4f6-4f72-99ea-b9321995dd6f admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:03.366980 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f279562-dd3e-47bf-8471-42a24dfb06dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:03.367649 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f279562-dd3e-47bf-8471-42a24dfb06dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:03.367935 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f279562-dd3e-47bf-8471-42a24dfb06dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:03.368242 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f279562-dd3e-47bf-8471-42a24dfb06dc admin admin] PATH_INFO: `/v3/projects/0bfd3d50672844dba24e7abc2552fc02/users/f591a2da8051467c9f8858e3a7047887/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:03.396640 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f279562-dd3e-47bf-8471-42a24dfb06dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bfd3d50672844dba24e7abc2552fc02, user_id=f591a2da8051467c9f8858e3a7047887, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:03.400253 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f279562-dd3e-47bf-8471-42a24dfb06dc admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:03.498799 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 955/1885] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:13:03 2022] PUT /identity/v3/projects/0bfd3d50672844dba24e7abc2552fc02/users/f591a2da8051467c9f8858e3a7047887/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:13:03.510083 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3224a249-f552-4e41-9c7c-3ed84e1d1476 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:03.510561 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3224a249-f552-4e41-9c7c-3ed84e1d1476 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:03.511220 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3224a249-f552-4e41-9c7c-3ed84e1d1476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:03.540548 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-3224a249-f552-4e41-9c7c-3ed84e1d1476 None None] MFA Rules not processed for user `f591a2da8051467c9f8858e3a7047887`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:13:03.541798 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-3224a249-f552-4e41-9c7c-3ed84e1d1476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:03.560944 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:13:03.608778 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 931/1886] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:13:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:13:03.613341 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f279562-dd3e-47bf-8471-42a24dfb06dc admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:03.679475 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f279562-dd3e-47bf-8471-42a24dfb06dc admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:03.684168 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a59da60-1bf0-4cd0-85bc-32bc967a95de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:03.684781 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a59da60-1bf0-4cd0-85bc-32bc967a95de admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:03.685066 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a59da60-1bf0-4cd0-85bc-32bc967a95de admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:03.685343 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a59da60-1bf0-4cd0-85bc-32bc967a95de admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:03.685974 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a59da60-1bf0-4cd0-85bc-32bc967a95de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:03.687950 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a59da60-1bf0-4cd0-85bc-32bc967a95de admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:03.785556 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 956/1887] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:13:03 2022] POST /identity/v3/projects => generated 377 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 17:13:03.789652 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3224a249-f552-4e41-9c7c-3ed84e1d1476 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:03.800512 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3224a249-f552-4e41-9c7c-3ed84e1d1476 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:03.857035 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7820836-f1b5-4c2f-a9f1-585736034d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:03.857675 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7820836-f1b5-4c2f-a9f1-585736034d53 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:03.857956 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7820836-f1b5-4c2f-a9f1-585736034d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:03.858228 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7820836-f1b5-4c2f-a9f1-585736034d53 admin admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:03.858838 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7820836-f1b5-4c2f-a9f1-585736034d53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:03.860786 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7820836-f1b5-4c2f-a9f1-585736034d53 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:03.949487 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 932/1888] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:13:03 2022] POST /identity/v3/users => generated 289 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:13:03.950712 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a59da60-1bf0-4cd0-85bc-32bc967a95de admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:03.962537 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a59da60-1bf0-4cd0-85bc-32bc967a95de admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:03.967963 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d39a766-2819-482e-9d95-fffd505c5b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:03.968578 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d39a766-2819-482e-9d95-fffd505c5b4c admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:03.968875 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d39a766-2819-482e-9d95-fffd505c5b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:03.969135 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d39a766-2819-482e-9d95-fffd505c5b4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:03.969795 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d39a766-2819-482e-9d95-fffd505c5b4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:03.971421 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d39a766-2819-482e-9d95-fffd505c5b4c admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:03.983177 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 957/1889] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:13:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:13:03.987265 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7820836-f1b5-4c2f-a9f1-585736034d53 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:03.997893 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7820836-f1b5-4c2f-a9f1-585736034d53 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:04.002548 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864ff292-95d4-49e6-8318-c03f9c612221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:04.003186 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864ff292-95d4-49e6-8318-c03f9c612221 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:04.003463 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864ff292-95d4-49e6-8318-c03f9c612221 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:04.003736 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864ff292-95d4-49e6-8318-c03f9c612221 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:04.004369 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864ff292-95d4-49e6-8318-c03f9c612221 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:04.006715 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864ff292-95d4-49e6-8318-c03f9c612221 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:04.016410 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 933/1890] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:13:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:13:04.020577 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d39a766-2819-482e-9d95-fffd505c5b4c admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:04.031234 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d39a766-2819-482e-9d95-fffd505c5b4c admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:04.035895 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762c9c2-5242-4c2e-8388-8e6b5bbc0c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:04.036554 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762c9c2-5242-4c2e-8388-8e6b5bbc0c0f admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:04.036866 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762c9c2-5242-4c2e-8388-8e6b5bbc0c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:04.037146 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762c9c2-5242-4c2e-8388-8e6b5bbc0c0f admin admin] PATH_INFO: `/v3/projects/3a5bf3b271d24af89238572c4aa250c1/users/4cc639a2fef5418695c4070e102269a1/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:04.053085 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f762c9c2-5242-4c2e-8388-8e6b5bbc0c0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a5bf3b271d24af89238572c4aa250c1, user_id=4cc639a2fef5418695c4070e102269a1, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:04.055963 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f762c9c2-5242-4c2e-8388-8e6b5bbc0c0f admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:04.125190 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:13:04.129834 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 958/1891] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:13:04 2022] PUT /identity/v3/projects/3a5bf3b271d24af89238572c4aa250c1/users/4cc639a2fef5418695c4070e102269a1/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:13:04.140807 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f656196-f709-450d-bd79-51fb3521e362 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:04.141166 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f656196-f709-450d-bd79-51fb3521e362 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:04.141451 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f656196-f709-450d-bd79-51fb3521e362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:04.169193 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-8f656196-f709-450d-bd79-51fb3521e362 None None] MFA Rules not processed for user `4cc639a2fef5418695c4070e102269a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:13:04.171666 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-8f656196-f709-450d-bd79-51fb3521e362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:04.240576 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 934/1892] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:13:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:13:04.245473 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762c9c2-5242-4c2e-8388-8e6b5bbc0c0f admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:04.313261 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762c9c2-5242-4c2e-8388-8e6b5bbc0c0f admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:04.319694 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6871ea-16d8-49dd-a166-d6501726375d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:04.320391 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6871ea-16d8-49dd-a166-d6501726375d admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:04.320745 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6871ea-16d8-49dd-a166-d6501726375d admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:04.321357 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6871ea-16d8-49dd-a166-d6501726375d admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:04.322089 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6871ea-16d8-49dd-a166-d6501726375d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:04.324316 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6871ea-16d8-49dd-a166-d6501726375d admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:04.421076 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 959/1893] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:13:04 2022] POST /identity/v3/projects => generated 377 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:13:04.422260 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f656196-f709-450d-bd79-51fb3521e362 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:04.435564 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f656196-f709-450d-bd79-51fb3521e362 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:04.471942 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:13:04.534210 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31b022b-0976-485e-914a-a998f75d8773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:04.535283 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31b022b-0976-485e-914a-a998f75d8773 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:04.535674 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31b022b-0976-485e-914a-a998f75d8773 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:04.536111 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31b022b-0976-485e-914a-a998f75d8773 admin admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:04.537536 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31b022b-0976-485e-914a-a998f75d8773 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:04.539813 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31b022b-0976-485e-914a-a998f75d8773 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:04.660974 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 935/1894] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:13:04 2022] POST /identity/v3/users => generated 295 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:13:04.662297 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6871ea-16d8-49dd-a166-d6501726375d admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:04.681775 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6871ea-16d8-49dd-a166-d6501726375d admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:04.704106 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665852bd-dbf5-4cd5-9202-fb3e17403c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:04.704865 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665852bd-dbf5-4cd5-9202-fb3e17403c19 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:04.705213 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665852bd-dbf5-4cd5-9202-fb3e17403c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:04.705581 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665852bd-dbf5-4cd5-9202-fb3e17403c19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:04.706738 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665852bd-dbf5-4cd5-9202-fb3e17403c19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:04.712982 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665852bd-dbf5-4cd5-9202-fb3e17403c19 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:04.747832 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 960/1895] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:13:04 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:13:04.749762 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31b022b-0976-485e-914a-a998f75d8773 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:04.776204 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31b022b-0976-485e-914a-a998f75d8773 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:04.790561 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21f4097-99f3-4451-9a26-8256c41a24d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:04.791582 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21f4097-99f3-4451-9a26-8256c41a24d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:04.791928 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21f4097-99f3-4451-9a26-8256c41a24d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:04.792510 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21f4097-99f3-4451-9a26-8256c41a24d3 admin admin] PATH_INFO: `/v3/projects/cd217e6ee3ce4331afa78380c4a17d56/users/b2396e1630384daa9c1843b5c564514e/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:04.814739 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21f4097-99f3-4451-9a26-8256c41a24d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd217e6ee3ce4331afa78380c4a17d56, user_id=b2396e1630384daa9c1843b5c564514e, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:04.818731 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21f4097-99f3-4451-9a26-8256c41a24d3 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:04.951437 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 936/1896] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:13:04 2022] PUT /identity/v3/projects/cd217e6ee3ce4331afa78380c4a17d56/users/b2396e1630384daa9c1843b5c564514e/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:13:04.960169 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665852bd-dbf5-4cd5-9202-fb3e17403c19 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:04.975871 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665852bd-dbf5-4cd5-9202-fb3e17403c19 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:05.058581 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067d4229-9910-4cc6-bbe9-1bbdabfc2c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:05.059233 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d4229-9910-4cc6-bbe9-1bbdabfc2c65 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:05.059520 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d4229-9910-4cc6-bbe9-1bbdabfc2c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:05.059798 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d4229-9910-4cc6-bbe9-1bbdabfc2c65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:05.060437 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067d4229-9910-4cc6-bbe9-1bbdabfc2c65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:05.062133 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067d4229-9910-4cc6-bbe9-1bbdabfc2c65 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:05.072983 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 961/1897] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:13:04 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:13:05.078623 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21f4097-99f3-4451-9a26-8256c41a24d3 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:05.090890 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21f4097-99f3-4451-9a26-8256c41a24d3 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:05.096008 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af5228a-e68f-4443-b4e3-bc1e91cc219f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:05.096704 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af5228a-e68f-4443-b4e3-bc1e91cc219f admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:05.096982 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af5228a-e68f-4443-b4e3-bc1e91cc219f admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:05.097255 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af5228a-e68f-4443-b4e3-bc1e91cc219f admin admin] PATH_INFO: `/v3/projects/cd217e6ee3ce4331afa78380c4a17d56/users/b2396e1630384daa9c1843b5c564514e/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:05.099216 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af5228a-e68f-4443-b4e3-bc1e91cc219f admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd217e6ee3ce4331afa78380c4a17d56, user_id=b2396e1630384daa9c1843b5c564514e, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:05.102850 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af5228a-e68f-4443-b4e3-bc1e91cc219f admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:05.113825 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 937/1898] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:13:05 2022] PUT /identity/v3/projects/cd217e6ee3ce4331afa78380c4a17d56/users/b2396e1630384daa9c1843b5c564514e/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:13:05.126167 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81039f3-47a4-47d2-b9b8-5edd693b74de None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:05.131755 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81039f3-47a4-47d2-b9b8-5edd693b74de None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:05.131755 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81039f3-47a4-47d2-b9b8-5edd693b74de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:05.156945 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-d81039f3-47a4-47d2-b9b8-5edd693b74de None None] MFA Rules not processed for user `b2396e1630384daa9c1843b5c564514e`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:13:05.158459 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-d81039f3-47a4-47d2-b9b8-5edd693b74de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:05.223665 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:13:05.234562 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 962/1899] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:13:05 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:13:05.246823 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af5228a-e68f-4443-b4e3-bc1e91cc219f admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:05.249469 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-0af5228a-e68f-4443-b4e3-bc1e91cc219f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:05.332927 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af5228a-e68f-4443-b4e3-bc1e91cc219f admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:05.338621 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d8fc35-6047-469f-9774-cac906dbc406 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:05.339324 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d8fc35-6047-469f-9774-cac906dbc406 service cinder] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:05.339616 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d8fc35-6047-469f-9774-cac906dbc406 service cinder] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:05.340017 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d8fc35-6047-469f-9774-cac906dbc406 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:05.425101 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d8fc35-6047-469f-9774-cac906dbc406 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:05.426557 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d8fc35-6047-469f-9774-cac906dbc406 service cinder] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:05.436185 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 938/1900] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:13:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:13:06.726967 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da85c37-d16b-4fe9-ac58-c14349d479be None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:06.727484 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da85c37-d16b-4fe9-ac58-c14349d479be None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:06.727587 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da85c37-d16b-4fe9-ac58-c14349d479be None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:06.729016 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 963/1901] 10.222.0.82 () {54 vars in 921 bytes} [Thu Aug 18 17:13:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:13:06.757647 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee245d29-3beb-48b2-a0e2-473ac91d5774 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:06.758203 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee245d29-3beb-48b2-a0e2-473ac91d5774 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:06.758203 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee245d29-3beb-48b2-a0e2-473ac91d5774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:06.778337 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-ee245d29-3beb-48b2-a0e2-473ac91d5774 None None] MFA Rules not processed for user `55723c756cae4e7ab636054db48de02d`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:13:06.779598 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-ee245d29-3beb-48b2-a0e2-473ac91d5774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:06.815570 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:13:06.854949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 939/1902] 10.222.0.82 () {58 vars in 1018 bytes} [Thu Aug 18 17:13:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:13:07.019006 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da85c37-d16b-4fe9-ac58-c14349d479be None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:07.020816 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-3da85c37-d16b-4fe9-ac58-c14349d479be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:07.039516 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da85c37-d16b-4fe9-ac58-c14349d479be None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:07.045586 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a645cdd-9f79-47cb-aa44-4e415d38019c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55723c756cae4e7ab636054db48de02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:07.046213 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a645cdd-9f79-47cb-aa44-4e415d38019c service nova] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:07.046481 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a645cdd-9f79-47cb-aa44-4e415d38019c service nova] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:07.046750 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a645cdd-9f79-47cb-aa44-4e415d38019c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:07.056830 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a645cdd-9f79-47cb-aa44-4e415d38019c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:07.058437 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a645cdd-9f79-47cb-aa44-4e415d38019c service nova] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:07.072008 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 964/1903] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:13:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:13:12.325430 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee245d29-3beb-48b2-a0e2-473ac91d5774 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:12.335275 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee245d29-3beb-48b2-a0e2-473ac91d5774 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:12.339902 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88ae924-00f7-4889-a7fe-f7cce1c8b415 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:12.341098 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ae924-00f7-4889-a7fe-f7cce1c8b415 service cinder] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:12.341098 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ae924-00f7-4889-a7fe-f7cce1c8b415 service cinder] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:12.346280 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ae924-00f7-4889-a7fe-f7cce1c8b415 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:12.399821 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88ae924-00f7-4889-a7fe-f7cce1c8b415 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:12.401350 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88ae924-00f7-4889-a7fe-f7cce1c8b415 service cinder] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:12.410021 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 940/1904] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:13:12 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 17:13:14.861083 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a645cdd-9f79-47cb-aa44-4e415d38019c service nova] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:14.862490 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-6a645cdd-9f79-47cb-aa44-4e415d38019c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:14.874751 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a645cdd-9f79-47cb-aa44-4e415d38019c service nova] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:14.878309 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12d9b8f-2662-4097-a5ee-de6f00f823c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:14.878916 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12d9b8f-2662-4097-a5ee-de6f00f823c6 service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:14.879200 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12d9b8f-2662-4097-a5ee-de6f00f823c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:14.879468 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12d9b8f-2662-4097-a5ee-de6f00f823c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:14.889001 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12d9b8f-2662-4097-a5ee-de6f00f823c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:14.890395 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12d9b8f-2662-4097-a5ee-de6f00f823c6 service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:14.900065 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 965/1905] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:13:14 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:13:22.406075 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=43 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:13:24.352633 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f232e5ae-b508-46b4-a54d-bf31cce0ec93 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:24.353314 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f232e5ae-b508-46b4-a54d-bf31cce0ec93 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:24.353389 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f232e5ae-b508-46b4-a54d-bf31cce0ec93 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:24.355429 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 941/1906] 10.222.0.82 () {54 vars in 921 bytes} [Thu Aug 18 17:13:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:13:24.380860 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd06ee-8a35-4fe2-a156-c8a086f4c041 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:24.381391 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd06ee-8a35-4fe2-a156-c8a086f4c041 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:24.381506 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd06ee-8a35-4fe2-a156-c8a086f4c041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:24.405004 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-27fd06ee-8a35-4fe2-a156-c8a086f4c041 None None] MFA Rules not processed for user `55723c756cae4e7ab636054db48de02d`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:13:24.406218 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-27fd06ee-8a35-4fe2-a156-c8a086f4c041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:24.418413 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 966/1907] 10.222.0.82 () {58 vars in 1018 bytes} [Thu Aug 18 17:13:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:13:24.436148 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f232e5ae-b508-46b4-a54d-bf31cce0ec93 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:24.437875 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-f232e5ae-b508-46b4-a54d-bf31cce0ec93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:24.460788 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f232e5ae-b508-46b4-a54d-bf31cce0ec93 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:24.468430 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5755a9-f5aa-4c4e-aad0-6d56b5bb0f12 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55723c756cae4e7ab636054db48de02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:24.469206 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5755a9-f5aa-4c4e-aad0-6d56b5bb0f12 service nova] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:24.470034 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5755a9-f5aa-4c4e-aad0-6d56b5bb0f12 service nova] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:24.470467 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5755a9-f5aa-4c4e-aad0-6d56b5bb0f12 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:24.486086 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5755a9-f5aa-4c4e-aad0-6d56b5bb0f12 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:24.487803 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5755a9-f5aa-4c4e-aad0-6d56b5bb0f12 service nova] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:24.499797 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 942/1908] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:13:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:13:24.909993 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd06ee-8a35-4fe2-a156-c8a086f4c041 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:24.911651 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-27fd06ee-8a35-4fe2-a156-c8a086f4c041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:24.976345 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd06ee-8a35-4fe2-a156-c8a086f4c041 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:24.980901 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddd664b-d999-478c-b739-b11354c85ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:24.981536 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd664b-d999-478c-b739-b11354c85ed7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:24.981817 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd664b-d999-478c-b739-b11354c85ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:24.982105 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd664b-d999-478c-b739-b11354c85ed7 admin admin] PATH_INFO: `/v3/users/2f1c9be4b5a34d6b89ca27d34e4e425c` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:24.983245 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddd664b-d999-478c-b739-b11354c85ed7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f1c9be4b5a34d6b89ca27d34e4e425c)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:24.985158 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddd664b-d999-478c-b739-b11354c85ed7 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:25.152197 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-9ddd664b-d999-478c-b739-b11354c85ed7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f1c9be4b5a34d6b89ca27d34e4e425c'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:25.242230 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-9ddd664b-d999-478c-b739-b11354c85ed7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f1c9be4b5a34d6b89ca27d34e4e425c'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:25.243070 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-9ddd664b-d999-478c-b739-b11354c85ed7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f1c9be4b5a34d6b89ca27d34e4e425c'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:25.243704 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-9ddd664b-d999-478c-b739-b11354c85ed7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f1c9be4b5a34d6b89ca27d34e4e425c'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:25.256450 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-9ddd664b-d999-478c-b739-b11354c85ed7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f1c9be4b5a34d6b89ca27d34e4e425c'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:25.281453 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 967/1909] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:13:24 2022] DELETE /identity/v3/users/2f1c9be4b5a34d6b89ca27d34e4e425c => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:13:25.285963 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b61b56-7be9-4dd4-bccc-ba8617e5bf5d None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:25.286266 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b61b56-7be9-4dd4-bccc-ba8617e5bf5d None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:25.286511 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b61b56-7be9-4dd4-bccc-ba8617e5bf5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:25.323234 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-81b61b56-7be9-4dd4-bccc-ba8617e5bf5d None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:13:25.324547 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-81b61b56-7be9-4dd4-bccc-ba8617e5bf5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:25.415192 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 943/1910] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:13:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:13:25.429421 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddd664b-d999-478c-b739-b11354c85ed7 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:25.431077 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-9ddd664b-d999-478c-b739-b11354c85ed7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:25.486650 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:13:25.499501 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddd664b-d999-478c-b739-b11354c85ed7 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:25.503341 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e781fed0-e634-4abc-ba97-c1b1d302177f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:25.504068 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e781fed0-e634-4abc-ba97-c1b1d302177f service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:25.504351 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e781fed0-e634-4abc-ba97-c1b1d302177f service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:25.504632 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e781fed0-e634-4abc-ba97-c1b1d302177f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:25.513935 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e781fed0-e634-4abc-ba97-c1b1d302177f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:25.515473 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e781fed0-e634-4abc-ba97-c1b1d302177f service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:25.543419 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 968/1911] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:13:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:13:25.973047 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b61b56-7be9-4dd4-bccc-ba8617e5bf5d None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:25.975176 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-81b61b56-7be9-4dd4-bccc-ba8617e5bf5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:25.987397 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b61b56-7be9-4dd4-bccc-ba8617e5bf5d None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:25.992423 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10d89a1-4201-463c-b06f-f7aa1cf61586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:25.993056 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10d89a1-4201-463c-b06f-f7aa1cf61586 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:25.993364 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10d89a1-4201-463c-b06f-f7aa1cf61586 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:25.993636 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10d89a1-4201-463c-b06f-f7aa1cf61586 admin admin] PATH_INFO: `/v3/projects/65eaaaeb789f43bca77172f33fa7918a` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:25.996500 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10d89a1-4201-463c-b06f-f7aa1cf61586 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65eaaaeb789f43bca77172f33fa7918a)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:25.998434 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10d89a1-4201-463c-b06f-f7aa1cf61586 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:26.102292 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-c10d89a1-4201-463c-b06f-f7aa1cf61586 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65eaaaeb789f43bca77172f33fa7918a', 'request_id': 'req-c10d89a1-4201-463c-b06f-f7aa1cf61586'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:26.190672 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-c10d89a1-4201-463c-b06f-f7aa1cf61586 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65eaaaeb789f43bca77172f33fa7918a', 'request_id': 'req-c10d89a1-4201-463c-b06f-f7aa1cf61586'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:26.197378 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-c10d89a1-4201-463c-b06f-f7aa1cf61586 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65eaaaeb789f43bca77172f33fa7918a', 'request_id': 'req-c10d89a1-4201-463c-b06f-f7aa1cf61586'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:26.209678 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-c10d89a1-4201-463c-b06f-f7aa1cf61586 admin admin] The token cache is being invalidate because project 65eaaaeb789f43bca77172f33fa7918a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81650) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:13:26.209967 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-c10d89a1-4201-463c-b06f-f7aa1cf61586 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:26.210621 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-c10d89a1-4201-463c-b06f-f7aa1cf61586 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:26.212121 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 944/1912] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:13:25 2022] DELETE /identity/v3/projects/65eaaaeb789f43bca77172f33fa7918a => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:13:26.216088 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e781fed0-e634-4abc-ba97-c1b1d302177f service neutron] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:26.234523 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e781fed0-e634-4abc-ba97-c1b1d302177f service neutron] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:26.281755 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598bb14f-d248-4050-af76-e145a3d59bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:26.282476 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598bb14f-d248-4050-af76-e145a3d59bbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:26.282758 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598bb14f-d248-4050-af76-e145a3d59bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:26.283071 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598bb14f-d248-4050-af76-e145a3d59bbb admin admin] PATH_INFO: `/v3/users/3b0a67092cef4e9ca9bea3694007703e` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:26.284220 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598bb14f-d248-4050-af76-e145a3d59bbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b0a67092cef4e9ca9bea3694007703e)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:26.286343 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598bb14f-d248-4050-af76-e145a3d59bbb admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:26.487498 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-598bb14f-d248-4050-af76-e145a3d59bbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b0a67092cef4e9ca9bea3694007703e'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:26.571914 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-598bb14f-d248-4050-af76-e145a3d59bbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b0a67092cef4e9ca9bea3694007703e'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:26.572624 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-598bb14f-d248-4050-af76-e145a3d59bbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b0a67092cef4e9ca9bea3694007703e'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:26.573317 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-598bb14f-d248-4050-af76-e145a3d59bbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b0a67092cef4e9ca9bea3694007703e'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:26.582116 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-598bb14f-d248-4050-af76-e145a3d59bbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b0a67092cef4e9ca9bea3694007703e'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:26.602749 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 969/1913] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:13:26 2022] DELETE /identity/v3/users/3b0a67092cef4e9ca9bea3694007703e => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:13:27.095282 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10d89a1-4201-463c-b06f-f7aa1cf61586 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:27.096739 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-c10d89a1-4201-463c-b06f-f7aa1cf61586 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:27.110338 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10d89a1-4201-463c-b06f-f7aa1cf61586 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:27.167015 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b991e03d-3182-422d-8966-d61551659c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:27.167688 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b991e03d-3182-422d-8966-d61551659c38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:27.168011 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b991e03d-3182-422d-8966-d61551659c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:27.168295 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b991e03d-3182-422d-8966-d61551659c38 admin admin] PATH_INFO: `/v3/projects/bc4d0439de81404581013e253274acc7` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:27.169726 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b991e03d-3182-422d-8966-d61551659c38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc4d0439de81404581013e253274acc7)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:27.171760 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b991e03d-3182-422d-8966-d61551659c38 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:27.183653 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:13:27.288569 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-b991e03d-3182-422d-8966-d61551659c38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc4d0439de81404581013e253274acc7', 'request_id': 'req-b991e03d-3182-422d-8966-d61551659c38'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:27.344983 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-b991e03d-3182-422d-8966-d61551659c38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc4d0439de81404581013e253274acc7', 'request_id': 'req-b991e03d-3182-422d-8966-d61551659c38'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:27.351324 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-b991e03d-3182-422d-8966-d61551659c38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc4d0439de81404581013e253274acc7', 'request_id': 'req-b991e03d-3182-422d-8966-d61551659c38'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:27.362364 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-b991e03d-3182-422d-8966-d61551659c38 admin admin] The token cache is being invalidate because project bc4d0439de81404581013e253274acc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81650) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:13:27.362654 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-b991e03d-3182-422d-8966-d61551659c38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:27.363476 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-b991e03d-3182-422d-8966-d61551659c38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:27.364971 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 945/1914] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:13:27 2022] DELETE /identity/v3/projects/bc4d0439de81404581013e253274acc7 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:13:27.372390 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ba408-b11c-4c32-b400-52d86f332511 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:27.372681 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ba408-b11c-4c32-b400-52d86f332511 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:27.372981 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ba408-b11c-4c32-b400-52d86f332511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:27.392368 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-7c9ba408-b11c-4c32-b400-52d86f332511 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:13:27.393672 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-7c9ba408-b11c-4c32-b400-52d86f332511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:27.458371 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 970/1915] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:13:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:13:27.465967 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6768b925-e4b7-4915-9038-1d60e9ee00f9 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:27.466253 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6768b925-e4b7-4915-9038-1d60e9ee00f9 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:27.466527 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6768b925-e4b7-4915-9038-1d60e9ee00f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:27.486124 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-6768b925-e4b7-4915-9038-1d60e9ee00f9 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:13:27.487359 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-6768b925-e4b7-4915-9038-1d60e9ee00f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:27.496133 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 946/1916] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:13:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:13:27.500301 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9ba408-b11c-4c32-b400-52d86f332511 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:27.506704 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:13:27.510581 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9ba408-b11c-4c32-b400-52d86f332511 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:27.515316 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2969c8e-f074-45f0-b856-b2e4f03e0731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:27.515920 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2969c8e-f074-45f0-b856-b2e4f03e0731 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:27.517495 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2969c8e-f074-45f0-b856-b2e4f03e0731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:27.517495 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2969c8e-f074-45f0-b856-b2e4f03e0731 admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:27.517495 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2969c8e-f074-45f0-b856-b2e4f03e0731 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:13:27.517495 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2969c8e-f074-45f0-b856-b2e4f03e0731 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:27.519727 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2969c8e-f074-45f0-b856-b2e4f03e0731 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:27.531910 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 971/1917] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:13:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 17:13:27.537222 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6768b925-e4b7-4915-9038-1d60e9ee00f9 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:27.546536 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6768b925-e4b7-4915-9038-1d60e9ee00f9 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:27.551198 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a146cb-47c9-48d5-8625-ea53d8d4cfba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:27.551792 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a146cb-47c9-48d5-8625-ea53d8d4cfba admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:27.552070 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a146cb-47c9-48d5-8625-ea53d8d4cfba admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:27.552340 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a146cb-47c9-48d5-8625-ea53d8d4cfba admin admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:27.552963 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a146cb-47c9-48d5-8625-ea53d8d4cfba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:27.554996 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a146cb-47c9-48d5-8625-ea53d8d4cfba admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:27.678312 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 947/1918] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:13:27 2022] POST /identity/v3/projects => generated 385 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:13:27.682378 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2969c8e-f074-45f0-b856-b2e4f03e0731 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:27.692939 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2969c8e-f074-45f0-b856-b2e4f03e0731 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:27.748953 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce3070c-0413-4050-b140-42531ba7f887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:27.749572 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce3070c-0413-4050-b140-42531ba7f887 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:27.749851 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce3070c-0413-4050-b140-42531ba7f887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:27.750132 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce3070c-0413-4050-b140-42531ba7f887 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:27.750765 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce3070c-0413-4050-b140-42531ba7f887 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:27.752714 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce3070c-0413-4050-b140-42531ba7f887 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:27.859693 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 972/1919] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:13:27 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:13:27.864651 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a146cb-47c9-48d5-8625-ea53d8d4cfba admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:27.875223 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a146cb-47c9-48d5-8625-ea53d8d4cfba admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:27.879915 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79f16a8-a8a9-4ded-8906-d358d5c96f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:27.880623 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f16a8-a8a9-4ded-8906-d358d5c96f41 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:27.880825 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f16a8-a8a9-4ded-8906-d358d5c96f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:27.881101 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f16a8-a8a9-4ded-8906-d358d5c96f41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:27.881742 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79f16a8-a8a9-4ded-8906-d358d5c96f41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:27.883402 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79f16a8-a8a9-4ded-8906-d358d5c96f41 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:27.894207 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 948/1920] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:13:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:13:27.898273 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce3070c-0413-4050-b140-42531ba7f887 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:27.908843 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce3070c-0413-4050-b140-42531ba7f887 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:27.916782 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b72daf-412c-4189-9d59-930f9a8d2466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:27.917442 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b72daf-412c-4189-9d59-930f9a8d2466 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:27.917724 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b72daf-412c-4189-9d59-930f9a8d2466 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:27.918069 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b72daf-412c-4189-9d59-930f9a8d2466 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:27.918668 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b72daf-412c-4189-9d59-930f9a8d2466 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:27.920987 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b72daf-412c-4189-9d59-930f9a8d2466 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:27.935426 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 973/1921] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:13:27 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:13:27.939274 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79f16a8-a8a9-4ded-8906-d358d5c96f41 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:27.950611 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79f16a8-a8a9-4ded-8906-d358d5c96f41 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:27.959930 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2294406a-5d5d-4ef9-bb20-7a61f0573cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:27.960649 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2294406a-5d5d-4ef9-bb20-7a61f0573cb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:27.960931 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2294406a-5d5d-4ef9-bb20-7a61f0573cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:27.961207 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2294406a-5d5d-4ef9-bb20-7a61f0573cb9 admin admin] PATH_INFO: `/v3/projects/7053619defb64841aa8e0a807c8f7367/users/afcaa4c9a2774a7388a6fcf90ba58449/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:27.980066 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2294406a-5d5d-4ef9-bb20-7a61f0573cb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7053619defb64841aa8e0a807c8f7367, user_id=afcaa4c9a2774a7388a6fcf90ba58449, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:27.982865 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2294406a-5d5d-4ef9-bb20-7a61f0573cb9 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:28.054526 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 949/1922] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:13:27 2022] PUT /identity/v3/projects/7053619defb64841aa8e0a807c8f7367/users/afcaa4c9a2774a7388a6fcf90ba58449/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:13:28.065975 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a75a8ee-9282-498e-a6df-ff632fc188d6 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:28.066240 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a75a8ee-9282-498e-a6df-ff632fc188d6 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:28.066513 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a75a8ee-9282-498e-a6df-ff632fc188d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:28.094316 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-0a75a8ee-9282-498e-a6df-ff632fc188d6 None None] MFA Rules not processed for user `afcaa4c9a2774a7388a6fcf90ba58449`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:13:28.095549 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-0a75a8ee-9282-498e-a6df-ff632fc188d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:28.161693 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 974/1923] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:13:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:13:28.172551 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2294406a-5d5d-4ef9-bb20-7a61f0573cb9 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:28.174300 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-2294406a-5d5d-4ef9-bb20-7a61f0573cb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:28.231286 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2294406a-5d5d-4ef9-bb20-7a61f0573cb9 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:28.234756 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6638642b-c69a-4d31-bed6-aa3c72e66fb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:28.235390 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6638642b-c69a-4d31-bed6-aa3c72e66fb5 service cinder] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:28.235672 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6638642b-c69a-4d31-bed6-aa3c72e66fb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:28.235960 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6638642b-c69a-4d31-bed6-aa3c72e66fb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:28.244435 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6638642b-c69a-4d31-bed6-aa3c72e66fb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:28.245876 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6638642b-c69a-4d31-bed6-aa3c72e66fb5 service cinder] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:28.254790 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 950/1924] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:13:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:13:32.920602 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a75a8ee-9282-498e-a6df-ff632fc188d6 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:32.922694 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-0a75a8ee-9282-498e-a6df-ff632fc188d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:33.040955 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a75a8ee-9282-498e-a6df-ff632fc188d6 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:33.047023 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9371234a-ab54-492e-9658-0a1a89183bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:33.047743 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9371234a-ab54-492e-9658-0a1a89183bd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:33.049261 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9371234a-ab54-492e-9658-0a1a89183bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:33.049768 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9371234a-ab54-492e-9658-0a1a89183bd8 admin admin] PATH_INFO: `/v3/users/f591a2da8051467c9f8858e3a7047887` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:33.051013 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9371234a-ab54-492e-9658-0a1a89183bd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f591a2da8051467c9f8858e3a7047887)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:33.053530 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9371234a-ab54-492e-9658-0a1a89183bd8 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:33.055696 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:13:33.375328 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-9371234a-ab54-492e-9658-0a1a89183bd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f591a2da8051467c9f8858e3a7047887'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:33.482882 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-9371234a-ab54-492e-9658-0a1a89183bd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f591a2da8051467c9f8858e3a7047887'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:33.483580 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-9371234a-ab54-492e-9658-0a1a89183bd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f591a2da8051467c9f8858e3a7047887'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:33.484166 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-9371234a-ab54-492e-9658-0a1a89183bd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f591a2da8051467c9f8858e3a7047887'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:33.493419 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-9371234a-ab54-492e-9658-0a1a89183bd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f591a2da8051467c9f8858e3a7047887'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:33.513383 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 975/1925] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:13:32 2022] DELETE /identity/v3/users/f591a2da8051467c9f8858e3a7047887 => generated 0 bytes in 593 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:13:33.518119 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c191b-0d6a-44f9-a1fc-d1beea23da26 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:33.519000 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c191b-0d6a-44f9-a1fc-d1beea23da26 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:33.519429 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c191b-0d6a-44f9-a1fc-d1beea23da26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:33.542221 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-ed3c191b-0d6a-44f9-a1fc-d1beea23da26 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:13:33.543793 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-ed3c191b-0d6a-44f9-a1fc-d1beea23da26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:33.580250 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:13:33.607277 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 951/1926] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:13:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:13:33.617678 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9371234a-ab54-492e-9658-0a1a89183bd8 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:33.619169 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-9371234a-ab54-492e-9658-0a1a89183bd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:33.680349 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9371234a-ab54-492e-9658-0a1a89183bd8 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:33.683970 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dba9bd-8049-47e0-9551-d52f973572cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:33.684583 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dba9bd-8049-47e0-9551-d52f973572cf service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:33.684861 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dba9bd-8049-47e0-9551-d52f973572cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:33.685137 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dba9bd-8049-47e0-9551-d52f973572cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:33.695403 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dba9bd-8049-47e0-9551-d52f973572cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:33.696837 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dba9bd-8049-47e0-9551-d52f973572cf service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:33.710660 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 976/1927] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:13:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:13:34.106123 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3c191b-0d6a-44f9-a1fc-d1beea23da26 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:34.108693 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-ed3c191b-0d6a-44f9-a1fc-d1beea23da26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:34.121857 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3c191b-0d6a-44f9-a1fc-d1beea23da26 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:34.127108 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7687f9c1-3cac-43d4-b35d-20fae91f0328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:34.131734 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7687f9c1-3cac-43d4-b35d-20fae91f0328 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:34.131734 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7687f9c1-3cac-43d4-b35d-20fae91f0328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:34.131734 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7687f9c1-3cac-43d4-b35d-20fae91f0328 admin admin] PATH_INFO: `/v3/projects/0bfd3d50672844dba24e7abc2552fc02` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:34.131734 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7687f9c1-3cac-43d4-b35d-20fae91f0328 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bfd3d50672844dba24e7abc2552fc02)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:34.131734 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7687f9c1-3cac-43d4-b35d-20fae91f0328 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:34.251266 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-7687f9c1-3cac-43d4-b35d-20fae91f0328 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bfd3d50672844dba24e7abc2552fc02', 'request_id': 'req-7687f9c1-3cac-43d4-b35d-20fae91f0328'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:34.359870 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-7687f9c1-3cac-43d4-b35d-20fae91f0328 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bfd3d50672844dba24e7abc2552fc02', 'request_id': 'req-7687f9c1-3cac-43d4-b35d-20fae91f0328'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:34.368580 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-7687f9c1-3cac-43d4-b35d-20fae91f0328 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bfd3d50672844dba24e7abc2552fc02', 'request_id': 'req-7687f9c1-3cac-43d4-b35d-20fae91f0328'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:34.387770 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-7687f9c1-3cac-43d4-b35d-20fae91f0328 admin admin] The token cache is being invalidate because project 0bfd3d50672844dba24e7abc2552fc02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81650) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:13:34.388652 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-7687f9c1-3cac-43d4-b35d-20fae91f0328 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:34.389261 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-7687f9c1-3cac-43d4-b35d-20fae91f0328 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:34.390863 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 952/1928] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:13:34 2022] DELETE /identity/v3/projects/0bfd3d50672844dba24e7abc2552fc02 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:13:34.395307 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dba9bd-8049-47e0-9551-d52f973572cf service neutron] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:34.409597 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dba9bd-8049-47e0-9551-d52f973572cf service neutron] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:34.492195 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c3c54b-434b-4102-aa69-3670a5eb5f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:34.492931 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c3c54b-434b-4102-aa69-3670a5eb5f92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:34.493263 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c3c54b-434b-4102-aa69-3670a5eb5f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:34.493599 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c3c54b-434b-4102-aa69-3670a5eb5f92 admin admin] PATH_INFO: `/v3/users/4cc639a2fef5418695c4070e102269a1` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:34.494816 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c3c54b-434b-4102-aa69-3670a5eb5f92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cc639a2fef5418695c4070e102269a1)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:34.497187 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c3c54b-434b-4102-aa69-3670a5eb5f92 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:34.524849 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:13:34.717442 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e2c3c54b-434b-4102-aa69-3670a5eb5f92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cc639a2fef5418695c4070e102269a1'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:34.811327 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e2c3c54b-434b-4102-aa69-3670a5eb5f92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cc639a2fef5418695c4070e102269a1'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:34.811327 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e2c3c54b-434b-4102-aa69-3670a5eb5f92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cc639a2fef5418695c4070e102269a1'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:34.811327 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e2c3c54b-434b-4102-aa69-3670a5eb5f92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cc639a2fef5418695c4070e102269a1'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:34.820617 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e2c3c54b-434b-4102-aa69-3670a5eb5f92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cc639a2fef5418695c4070e102269a1'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:34.846992 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 977/1929] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:13:34 2022] DELETE /identity/v3/users/4cc639a2fef5418695c4070e102269a1 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:13:35.306545 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7687f9c1-3cac-43d4-b35d-20fae91f0328 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:35.310761 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-7687f9c1-3cac-43d4-b35d-20fae91f0328 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:35.322592 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7687f9c1-3cac-43d4-b35d-20fae91f0328 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:35.381766 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2324059f-3eaf-4c32-880f-5ac6e6c417d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:35.382669 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2324059f-3eaf-4c32-880f-5ac6e6c417d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:35.382975 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2324059f-3eaf-4c32-880f-5ac6e6c417d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:35.383251 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2324059f-3eaf-4c32-880f-5ac6e6c417d2 admin admin] PATH_INFO: `/v3/projects/3a5bf3b271d24af89238572c4aa250c1` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:35.384378 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2324059f-3eaf-4c32-880f-5ac6e6c417d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a5bf3b271d24af89238572c4aa250c1)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:35.386284 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2324059f-3eaf-4c32-880f-5ac6e6c417d2 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:35.494786 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-2324059f-3eaf-4c32-880f-5ac6e6c417d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a5bf3b271d24af89238572c4aa250c1', 'request_id': 'req-2324059f-3eaf-4c32-880f-5ac6e6c417d2'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:35.567341 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-2324059f-3eaf-4c32-880f-5ac6e6c417d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a5bf3b271d24af89238572c4aa250c1', 'request_id': 'req-2324059f-3eaf-4c32-880f-5ac6e6c417d2'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:35.576523 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-2324059f-3eaf-4c32-880f-5ac6e6c417d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a5bf3b271d24af89238572c4aa250c1', 'request_id': 'req-2324059f-3eaf-4c32-880f-5ac6e6c417d2'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:35.577688 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:13:35.589031 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-2324059f-3eaf-4c32-880f-5ac6e6c417d2 admin admin] The token cache is being invalidate because project 3a5bf3b271d24af89238572c4aa250c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81650) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:13:35.589314 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-2324059f-3eaf-4c32-880f-5ac6e6c417d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:35.589969 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-2324059f-3eaf-4c32-880f-5ac6e6c417d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:35.594676 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 953/1930] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:13:35 2022] DELETE /identity/v3/projects/3a5bf3b271d24af89238572c4aa250c1 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:13:35.595981 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c3c54b-434b-4102-aa69-3670a5eb5f92 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:35.597609 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-e2c3c54b-434b-4102-aa69-3670a5eb5f92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:35.611553 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c3c54b-434b-4102-aa69-3670a5eb5f92 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:35.678484 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afd2b4e-a096-49ac-bab9-0e7b952b6710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:35.679173 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afd2b4e-a096-49ac-bab9-0e7b952b6710 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:35.679459 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afd2b4e-a096-49ac-bab9-0e7b952b6710 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:35.679736 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afd2b4e-a096-49ac-bab9-0e7b952b6710 admin admin] PATH_INFO: `/v3/users/b2396e1630384daa9c1843b5c564514e` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:35.680903 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afd2b4e-a096-49ac-bab9-0e7b952b6710 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2396e1630384daa9c1843b5c564514e)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:35.683012 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afd2b4e-a096-49ac-bab9-0e7b952b6710 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:35.833103 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-9afd2b4e-a096-49ac-bab9-0e7b952b6710 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2396e1630384daa9c1843b5c564514e'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:35.875846 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-9afd2b4e-a096-49ac-bab9-0e7b952b6710 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2396e1630384daa9c1843b5c564514e'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:35.876469 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-9afd2b4e-a096-49ac-bab9-0e7b952b6710 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2396e1630384daa9c1843b5c564514e'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:35.877377 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-9afd2b4e-a096-49ac-bab9-0e7b952b6710 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2396e1630384daa9c1843b5c564514e'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:35.884523 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-9afd2b4e-a096-49ac-bab9-0e7b952b6710 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2396e1630384daa9c1843b5c564514e'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:35.901875 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 978/1931] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:13:35 2022] DELETE /identity/v3/users/b2396e1630384daa9c1843b5c564514e => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:13:36.305953 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2324059f-3eaf-4c32-880f-5ac6e6c417d2 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:36.307427 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-2324059f-3eaf-4c32-880f-5ac6e6c417d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:36.322207 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2324059f-3eaf-4c32-880f-5ac6e6c417d2 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:36.379979 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c1b1b2-9842-4eeb-9908-422e2a183b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:36.380624 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c1b1b2-9842-4eeb-9908-422e2a183b71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:36.380905 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c1b1b2-9842-4eeb-9908-422e2a183b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:36.381180 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c1b1b2-9842-4eeb-9908-422e2a183b71 admin admin] PATH_INFO: `/v3/projects/cd217e6ee3ce4331afa78380c4a17d56` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:36.382295 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c1b1b2-9842-4eeb-9908-422e2a183b71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd217e6ee3ce4331afa78380c4a17d56)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:36.384325 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c1b1b2-9842-4eeb-9908-422e2a183b71 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:36.487894 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-35c1b1b2-9842-4eeb-9908-422e2a183b71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd217e6ee3ce4331afa78380c4a17d56', 'request_id': 'req-35c1b1b2-9842-4eeb-9908-422e2a183b71'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:36.532666 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-35c1b1b2-9842-4eeb-9908-422e2a183b71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd217e6ee3ce4331afa78380c4a17d56', 'request_id': 'req-35c1b1b2-9842-4eeb-9908-422e2a183b71'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:36.538191 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-35c1b1b2-9842-4eeb-9908-422e2a183b71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd217e6ee3ce4331afa78380c4a17d56', 'request_id': 'req-35c1b1b2-9842-4eeb-9908-422e2a183b71'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:36.550064 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-35c1b1b2-9842-4eeb-9908-422e2a183b71 admin admin] The token cache is being invalidate because project cd217e6ee3ce4331afa78380c4a17d56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81650) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:13:36.550352 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-35c1b1b2-9842-4eeb-9908-422e2a183b71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:36.551009 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-35c1b1b2-9842-4eeb-9908-422e2a183b71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:36.552483 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 954/1932] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:13:36 2022] DELETE /identity/v3/projects/cd217e6ee3ce4331afa78380c4a17d56 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:13:36.557398 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7456ba0-ca52-4488-b4fb-3de1c29c067a None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:36.557692 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7456ba0-ca52-4488-b4fb-3de1c29c067a None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:36.557969 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7456ba0-ca52-4488-b4fb-3de1c29c067a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:36.577279 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-a7456ba0-ca52-4488-b4fb-3de1c29c067a None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:13:36.578487 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-a7456ba0-ca52-4488-b4fb-3de1c29c067a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:36.592005 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:13:36.638822 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 979/1933] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:13:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:13:36.646462 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a504486a-9ec8-4654-a486-ad57feb158a6 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:36.646748 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a504486a-9ec8-4654-a486-ad57feb158a6 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:36.647070 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a504486a-9ec8-4654-a486-ad57feb158a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:36.666969 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-a504486a-9ec8-4654-a486-ad57feb158a6 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:13:36.668170 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-a504486a-9ec8-4654-a486-ad57feb158a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:36.676279 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 955/1934] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:13:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:13:36.680448 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7456ba0-ca52-4488-b4fb-3de1c29c067a None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:36.689786 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7456ba0-ca52-4488-b4fb-3de1c29c067a None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:36.694430 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becedfda-a37a-4857-b46b-aa1d0c4df35d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:36.695073 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becedfda-a37a-4857-b46b-aa1d0c4df35d admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:36.695360 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becedfda-a37a-4857-b46b-aa1d0c4df35d admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:36.695632 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becedfda-a37a-4857-b46b-aa1d0c4df35d admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:36.696768 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becedfda-a37a-4857-b46b-aa1d0c4df35d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:13:36.697072 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becedfda-a37a-4857-b46b-aa1d0c4df35d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:36.698666 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becedfda-a37a-4857-b46b-aa1d0c4df35d admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:36.714326 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 980/1935] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:13:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 17:13:36.718559 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a504486a-9ec8-4654-a486-ad57feb158a6 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:36.737689 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a504486a-9ec8-4654-a486-ad57feb158a6 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:36.749625 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21b708d-a878-4e59-a178-ebcbcd2ae6ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:36.750184 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b708d-a878-4e59-a178-ebcbcd2ae6ce admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:36.750466 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b708d-a878-4e59-a178-ebcbcd2ae6ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:36.750741 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b708d-a878-4e59-a178-ebcbcd2ae6ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:36.753885 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21b708d-a878-4e59-a178-ebcbcd2ae6ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:36.755862 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21b708d-a878-4e59-a178-ebcbcd2ae6ce admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:36.863377 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 956/1936] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:13:36 2022] POST /identity/v3/projects => generated 383 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:13:36.864681 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becedfda-a37a-4857-b46b-aa1d0c4df35d admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:36.874972 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becedfda-a37a-4857-b46b-aa1d0c4df35d admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:36.939728 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0fab51-d814-41bd-bb45-6765aa5ed757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:36.940669 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0fab51-d814-41bd-bb45-6765aa5ed757 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:36.941149 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0fab51-d814-41bd-bb45-6765aa5ed757 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:36.941584 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0fab51-d814-41bd-bb45-6765aa5ed757 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:36.942380 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0fab51-d814-41bd-bb45-6765aa5ed757 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:36.944350 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0fab51-d814-41bd-bb45-6765aa5ed757 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:37.017187 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 981/1937] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:13:36 2022] POST /identity/v3/users => generated 292 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:13:37.026709 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21b708d-a878-4e59-a178-ebcbcd2ae6ce admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:37.041260 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21b708d-a878-4e59-a178-ebcbcd2ae6ce admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:37.041260 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43afd29-25fe-4afd-a5d9-a89fda1c9be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:37.041260 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43afd29-25fe-4afd-a5d9-a89fda1c9be8 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:37.041260 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43afd29-25fe-4afd-a5d9-a89fda1c9be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:37.042252 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43afd29-25fe-4afd-a5d9-a89fda1c9be8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:37.042252 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43afd29-25fe-4afd-a5d9-a89fda1c9be8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:37.042252 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43afd29-25fe-4afd-a5d9-a89fda1c9be8 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:37.053565 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 957/1938] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:13:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:13:37.058436 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0fab51-d814-41bd-bb45-6765aa5ed757 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:37.068367 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0fab51-d814-41bd-bb45-6765aa5ed757 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:37.073881 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44b7806-7ed1-449f-bbf2-5572aaf55c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:37.074483 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b7806-7ed1-449f-bbf2-5572aaf55c65 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:37.074761 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b7806-7ed1-449f-bbf2-5572aaf55c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:37.075070 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b7806-7ed1-449f-bbf2-5572aaf55c65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:37.075704 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44b7806-7ed1-449f-bbf2-5572aaf55c65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:37.077281 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44b7806-7ed1-449f-bbf2-5572aaf55c65 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:37.088414 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 982/1939] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:13:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:13:37.092513 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43afd29-25fe-4afd-a5d9-a89fda1c9be8 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:37.102449 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43afd29-25fe-4afd-a5d9-a89fda1c9be8 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:37.107249 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d76405-58a3-4118-beef-4700699f05c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:37.107898 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d76405-58a3-4118-beef-4700699f05c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:37.108178 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d76405-58a3-4118-beef-4700699f05c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:37.108453 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d76405-58a3-4118-beef-4700699f05c5 admin admin] PATH_INFO: `/v3/projects/4529421fbf2445259eb7dde6b2df831a/users/94d03f8b4756475ca1502e100203e947/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:37.123065 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d76405-58a3-4118-beef-4700699f05c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4529421fbf2445259eb7dde6b2df831a, user_id=94d03f8b4756475ca1502e100203e947, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:37.125735 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d76405-58a3-4118-beef-4700699f05c5 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:37.193419 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 958/1940] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:13:37 2022] PUT /identity/v3/projects/4529421fbf2445259eb7dde6b2df831a/users/94d03f8b4756475ca1502e100203e947/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:13:37.207308 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21c5f6f-0ea8-40bd-acfe-fcacfe4c75ea None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:37.207308 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21c5f6f-0ea8-40bd-acfe-fcacfe4c75ea None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:37.207308 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21c5f6f-0ea8-40bd-acfe-fcacfe4c75ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:37.235248 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-d21c5f6f-0ea8-40bd-acfe-fcacfe4c75ea None None] MFA Rules not processed for user `94d03f8b4756475ca1502e100203e947`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:13:37.236477 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-d21c5f6f-0ea8-40bd-acfe-fcacfe4c75ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:37.303364 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 983/1941] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:13:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:13:37.307844 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d76405-58a3-4118-beef-4700699f05c5 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:37.329246 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:13:37.370735 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d76405-58a3-4118-beef-4700699f05c5 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:37.377755 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45896e0-5de4-42d5-a7f3-ae8a5ff1cb30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:37.378412 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45896e0-5de4-42d5-a7f3-ae8a5ff1cb30 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:37.378701 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45896e0-5de4-42d5-a7f3-ae8a5ff1cb30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:37.379010 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45896e0-5de4-42d5-a7f3-ae8a5ff1cb30 admin admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:37.379632 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45896e0-5de4-42d5-a7f3-ae8a5ff1cb30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:37.381561 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45896e0-5de4-42d5-a7f3-ae8a5ff1cb30 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:37.505956 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 959/1942] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:13:37 2022] POST /identity/v3/projects => generated 383 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:13:37.510299 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21c5f6f-0ea8-40bd-acfe-fcacfe4c75ea None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:37.521298 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21c5f6f-0ea8-40bd-acfe-fcacfe4c75ea None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:37.533698 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:13:37.578153 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca6be69-1d20-4f1d-9050-91142134e289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:37.578693 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca6be69-1d20-4f1d-9050-91142134e289 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:37.579009 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca6be69-1d20-4f1d-9050-91142134e289 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:37.579283 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca6be69-1d20-4f1d-9050-91142134e289 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:37.579914 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca6be69-1d20-4f1d-9050-91142134e289 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:37.581871 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca6be69-1d20-4f1d-9050-91142134e289 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:37.699394 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 984/1943] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:13:37 2022] POST /identity/v3/users => generated 292 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:13:37.703469 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45896e0-5de4-42d5-a7f3-ae8a5ff1cb30 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:37.714007 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45896e0-5de4-42d5-a7f3-ae8a5ff1cb30 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:37.719828 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b83e45-0e59-49bb-8b44-47d4fee3f4ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:37.720853 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b83e45-0e59-49bb-8b44-47d4fee3f4ec admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:37.721356 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b83e45-0e59-49bb-8b44-47d4fee3f4ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:37.721811 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b83e45-0e59-49bb-8b44-47d4fee3f4ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:37.722958 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b83e45-0e59-49bb-8b44-47d4fee3f4ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:37.725145 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b83e45-0e59-49bb-8b44-47d4fee3f4ec admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:37.735668 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 960/1944] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:13:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:13:37.739612 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca6be69-1d20-4f1d-9050-91142134e289 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:37.750286 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca6be69-1d20-4f1d-9050-91142134e289 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:37.754845 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c0196-fd4e-489d-b7e3-077a03642801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:37.755465 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c0196-fd4e-489d-b7e3-077a03642801 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:37.755744 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c0196-fd4e-489d-b7e3-077a03642801 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:37.756014 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c0196-fd4e-489d-b7e3-077a03642801 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:37.756644 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040c0196-fd4e-489d-b7e3-077a03642801 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:37.758216 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040c0196-fd4e-489d-b7e3-077a03642801 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:37.772690 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 985/1945] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:13:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:13:37.773496 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b83e45-0e59-49bb-8b44-47d4fee3f4ec admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:37.788678 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b83e45-0e59-49bb-8b44-47d4fee3f4ec admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:37.793909 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc4b283-fd0f-45bd-a313-560e534bcfec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:37.794587 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4b283-fd0f-45bd-a313-560e534bcfec admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:37.794865 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4b283-fd0f-45bd-a313-560e534bcfec admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:37.795170 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4b283-fd0f-45bd-a313-560e534bcfec admin admin] PATH_INFO: `/v3/projects/6e4c0c3313ea490e948b7353d0cd3ce6/users/97144aca51ae4154b43febf8611e2b23/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:37.811140 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc4b283-fd0f-45bd-a313-560e534bcfec admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e4c0c3313ea490e948b7353d0cd3ce6, user_id=97144aca51ae4154b43febf8611e2b23, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:37.813853 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc4b283-fd0f-45bd-a313-560e534bcfec admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:37.879405 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 961/1946] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:13:37 2022] PUT /identity/v3/projects/6e4c0c3313ea490e948b7353d0cd3ce6/users/97144aca51ae4154b43febf8611e2b23/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:13:37.893531 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da99d3-dcdc-4eba-8077-d0c67aae0283 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:37.893821 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da99d3-dcdc-4eba-8077-d0c67aae0283 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:37.894105 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da99d3-dcdc-4eba-8077-d0c67aae0283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:37.923458 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-74da99d3-dcdc-4eba-8077-d0c67aae0283 None None] MFA Rules not processed for user `97144aca51ae4154b43febf8611e2b23`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:13:37.925608 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-74da99d3-dcdc-4eba-8077-d0c67aae0283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:38.002186 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 986/1947] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:13:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:13:38.007000 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc4b283-fd0f-45bd-a313-560e534bcfec admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:38.083571 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc4b283-fd0f-45bd-a313-560e534bcfec admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:38.091219 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dd3c2c-0348-4593-a27d-bd8ddaa789dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:38.091997 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dd3c2c-0348-4593-a27d-bd8ddaa789dd admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:38.092269 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dd3c2c-0348-4593-a27d-bd8ddaa789dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:38.092546 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dd3c2c-0348-4593-a27d-bd8ddaa789dd admin admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:38.093235 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dd3c2c-0348-4593-a27d-bd8ddaa789dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:38.095233 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dd3c2c-0348-4593-a27d-bd8ddaa789dd admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:38.195320 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 962/1948] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:13:38 2022] POST /identity/v3/projects => generated 383 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 17:13:38.200072 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74da99d3-dcdc-4eba-8077-d0c67aae0283 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:38.221822 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74da99d3-dcdc-4eba-8077-d0c67aae0283 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:38.279595 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5be935f-3e13-4de4-b425-63e475051548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:38.280269 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be935f-3e13-4de4-b425-63e475051548 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:38.280557 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be935f-3e13-4de4-b425-63e475051548 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:38.280894 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be935f-3e13-4de4-b425-63e475051548 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:38.281513 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5be935f-3e13-4de4-b425-63e475051548 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:38.283516 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5be935f-3e13-4de4-b425-63e475051548 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:38.309806 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dd3c2c-0348-4593-a27d-bd8ddaa789dd admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:38.322507 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-84dd3c2c-0348-4593-a27d-bd8ddaa789dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:38.325327 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dd3c2c-0348-4593-a27d-bd8ddaa789dd admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:38.331386 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6162f1d8-c1e4-456e-863a-c94e1489563a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:38.332013 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6162f1d8-c1e4-456e-863a-c94e1489563a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:38.332292 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6162f1d8-c1e4-456e-863a-c94e1489563a admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:38.332568 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6162f1d8-c1e4-456e-863a-c94e1489563a admin admin] PATH_INFO: `/v3/users/1b4be4e285ad4556800f36754998b2ce` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:38.333766 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6162f1d8-c1e4-456e-863a-c94e1489563a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b4be4e285ad4556800f36754998b2ce)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:38.335875 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6162f1d8-c1e4-456e-863a-c94e1489563a admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:38.377559 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 987/1949] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:13:38 2022] POST /identity/v3/users => generated 298 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:13:38.381556 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5be935f-3e13-4de4-b425-63e475051548 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:38.392896 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5be935f-3e13-4de4-b425-63e475051548 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:38.397689 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7522dd-fe8e-40b6-974a-f0c1dd179698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:38.398304 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7522dd-fe8e-40b6-974a-f0c1dd179698 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:38.398587 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7522dd-fe8e-40b6-974a-f0c1dd179698 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:38.398861 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7522dd-fe8e-40b6-974a-f0c1dd179698 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:38.400488 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7522dd-fe8e-40b6-974a-f0c1dd179698 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:38.402439 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7522dd-fe8e-40b6-974a-f0c1dd179698 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:38.412918 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 988/1950] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:13:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 17:13:38.417337 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7522dd-fe8e-40b6-974a-f0c1dd179698 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:38.422871 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:13:38.428084 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7522dd-fe8e-40b6-974a-f0c1dd179698 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:38.433557 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9235fc37-455e-4398-b756-3660efbc12a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:38.437443 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9235fc37-455e-4398-b756-3660efbc12a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:38.437443 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9235fc37-455e-4398-b756-3660efbc12a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:38.437443 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9235fc37-455e-4398-b756-3660efbc12a7 admin admin] PATH_INFO: `/v3/projects/7982c59fbbe4438aa53b45684e0fa5e3/users/65ed8341b4ce48418e6bb0374da39373/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:38.450774 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9235fc37-455e-4398-b756-3660efbc12a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7982c59fbbe4438aa53b45684e0fa5e3, user_id=65ed8341b4ce48418e6bb0374da39373, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:38.453668 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9235fc37-455e-4398-b756-3660efbc12a7 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:38.588551 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 989/1951] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:13:38 2022] PUT /identity/v3/projects/7982c59fbbe4438aa53b45684e0fa5e3/users/65ed8341b4ce48418e6bb0374da39373/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:13:38.590147 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9235fc37-455e-4398-b756-3660efbc12a7 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:38.628918 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-6162f1d8-c1e4-456e-863a-c94e1489563a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b4be4e285ad4556800f36754998b2ce'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:38.637125 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:13:38.660516 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-6162f1d8-c1e4-456e-863a-c94e1489563a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b4be4e285ad4556800f36754998b2ce'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:38.661820 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-6162f1d8-c1e4-456e-863a-c94e1489563a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b4be4e285ad4556800f36754998b2ce'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:38.686453 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9235fc37-455e-4398-b756-3660efbc12a7 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:38.691206 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c55113-a0ca-4f30-a656-b94073f45e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:38.691851 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c55113-a0ca-4f30-a656-b94073f45e00 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:38.692138 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c55113-a0ca-4f30-a656-b94073f45e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:38.692413 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c55113-a0ca-4f30-a656-b94073f45e00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:38.693080 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c55113-a0ca-4f30-a656-b94073f45e00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:38.694875 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c55113-a0ca-4f30-a656-b94073f45e00 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:38.706947 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 990/1952] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:13:38 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:13:38.709638 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c55113-a0ca-4f30-a656-b94073f45e00 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:38.719841 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c55113-a0ca-4f30-a656-b94073f45e00 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:38.724611 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba0e2f5-4703-403f-bba1-404f6d69a26b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:38.725273 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0e2f5-4703-403f-bba1-404f6d69a26b admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:38.725580 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0e2f5-4703-403f-bba1-404f6d69a26b admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:38.725861 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0e2f5-4703-403f-bba1-404f6d69a26b admin admin] PATH_INFO: `/v3/projects/7982c59fbbe4438aa53b45684e0fa5e3/users/65ed8341b4ce48418e6bb0374da39373/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:38.727915 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba0e2f5-4703-403f-bba1-404f6d69a26b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7982c59fbbe4438aa53b45684e0fa5e3, user_id=65ed8341b4ce48418e6bb0374da39373, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:38.730571 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba0e2f5-4703-403f-bba1-404f6d69a26b admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:38.738711 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-6162f1d8-c1e4-456e-863a-c94e1489563a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b4be4e285ad4556800f36754998b2ce'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:38.744476 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-6162f1d8-c1e4-456e-863a-c94e1489563a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b4be4e285ad4556800f36754998b2ce'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:38.744601 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 991/1953] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:13:38 2022] PUT /identity/v3/projects/7982c59fbbe4438aa53b45684e0fa5e3/users/65ed8341b4ce48418e6bb0374da39373/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:13:38.752408 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74fac1d-a23f-4165-9892-3ddda71bd1a7 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:38.752729 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74fac1d-a23f-4165-9892-3ddda71bd1a7 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:38.752992 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74fac1d-a23f-4165-9892-3ddda71bd1a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:38.755406 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 963/1954] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:13:38 2022] DELETE /identity/v3/users/1b4be4e285ad4556800f36754998b2ce => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:13:38.760499 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016cb712-9886-4d62-b40c-fa6a99cc946f None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:38.760789 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016cb712-9886-4d62-b40c-fa6a99cc946f None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:38.761066 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016cb712-9886-4d62-b40c-fa6a99cc946f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:38.783112 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-a74fac1d-a23f-4165-9892-3ddda71bd1a7 None None] MFA Rules not processed for user `65ed8341b4ce48418e6bb0374da39373`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:13:38.784280 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-016cb712-9886-4d62-b40c-fa6a99cc946f None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:13:38.784400 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-a74fac1d-a23f-4165-9892-3ddda71bd1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:38.785662 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-016cb712-9886-4d62-b40c-fa6a99cc946f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:38.847207 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 964/1955] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:13:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 17:13:38.857526 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016cb712-9886-4d62-b40c-fa6a99cc946f None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:38.858891 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-016cb712-9886-4d62-b40c-fa6a99cc946f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:38.860362 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 992/1956] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:13:38 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:13:38.874022 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74fac1d-a23f-4165-9892-3ddda71bd1a7 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:38.875633 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-a74fac1d-a23f-4165-9892-3ddda71bd1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:38.918392 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016cb712-9886-4d62-b40c-fa6a99cc946f None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:38.922363 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc626fa-7112-478a-9f0f-79b4e255878b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:38.923009 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc626fa-7112-478a-9f0f-79b4e255878b service neutron] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:38.923301 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc626fa-7112-478a-9f0f-79b4e255878b service neutron] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:38.923571 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc626fa-7112-478a-9f0f-79b4e255878b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:38.929468 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:13:38.933537 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc626fa-7112-478a-9f0f-79b4e255878b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:38.935056 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc626fa-7112-478a-9f0f-79b4e255878b service neutron] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:38.936094 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74fac1d-a23f-4165-9892-3ddda71bd1a7 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:38.946985 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 965/1957] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:13:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 17:13:38.947231 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf585b62-0c51-4887-b1ef-86d4598c22b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:38.948101 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf585b62-0c51-4887-b1ef-86d4598c22b3 service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:38.948565 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf585b62-0c51-4887-b1ef-86d4598c22b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:38.948992 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf585b62-0c51-4887-b1ef-86d4598c22b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:38.951223 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-cf585b62-0c51-4887-b1ef-86d4598c22b3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:39.016578 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf585b62-0c51-4887-b1ef-86d4598c22b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:39.018117 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf585b62-0c51-4887-b1ef-86d4598c22b3 service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:39.027900 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 993/1958] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:13:38 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 17:13:39.551432 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc626fa-7112-478a-9f0f-79b4e255878b service neutron] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:39.552891 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-7bc626fa-7112-478a-9f0f-79b4e255878b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:39.568154 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc626fa-7112-478a-9f0f-79b4e255878b service neutron] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:39.573336 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db99a378-9097-47c9-9ae3-6e20eb41c94f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:39.574182 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db99a378-9097-47c9-9ae3-6e20eb41c94f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:39.574650 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db99a378-9097-47c9-9ae3-6e20eb41c94f admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:39.574880 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db99a378-9097-47c9-9ae3-6e20eb41c94f admin admin] PATH_INFO: `/v3/projects/53ae2db4841d4c1da8b04af100612c64` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:39.576051 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db99a378-9097-47c9-9ae3-6e20eb41c94f admin admin] RBAC: Authorizing `identity:delete_project(project_id=53ae2db4841d4c1da8b04af100612c64)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:39.578732 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db99a378-9097-47c9-9ae3-6e20eb41c94f admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:39.708721 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-db99a378-9097-47c9-9ae3-6e20eb41c94f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53ae2db4841d4c1da8b04af100612c64', 'request_id': 'req-db99a378-9097-47c9-9ae3-6e20eb41c94f'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:39.806512 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-db99a378-9097-47c9-9ae3-6e20eb41c94f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53ae2db4841d4c1da8b04af100612c64', 'request_id': 'req-db99a378-9097-47c9-9ae3-6e20eb41c94f'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:39.815313 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-db99a378-9097-47c9-9ae3-6e20eb41c94f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53ae2db4841d4c1da8b04af100612c64', 'request_id': 'req-db99a378-9097-47c9-9ae3-6e20eb41c94f'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:39.816749 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:13:39.830460 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-db99a378-9097-47c9-9ae3-6e20eb41c94f admin admin] The token cache is being invalidate because project 53ae2db4841d4c1da8b04af100612c64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81650) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:13:39.830460 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-db99a378-9097-47c9-9ae3-6e20eb41c94f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:39.831100 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-db99a378-9097-47c9-9ae3-6e20eb41c94f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:39.843433 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 966/1959] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:13:39 2022] DELETE /identity/v3/projects/53ae2db4841d4c1da8b04af100612c64 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:13:39.844420 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf585b62-0c51-4887-b1ef-86d4598c22b3 service cinder] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:39.858126 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf585b62-0c51-4887-b1ef-86d4598c22b3 service cinder] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:39.923071 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6701be98-41ca-4b8d-a1a2-ccabb855377b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:39.924354 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701be98-41ca-4b8d-a1a2-ccabb855377b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:39.924354 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701be98-41ca-4b8d-a1a2-ccabb855377b admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:39.924855 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701be98-41ca-4b8d-a1a2-ccabb855377b admin admin] PATH_INFO: `/v3/users/951a1bc499fc4c2aa4f5d08fcf690ff5` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:39.926084 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6701be98-41ca-4b8d-a1a2-ccabb855377b admin admin] RBAC: Authorizing `identity:delete_user(user_id=951a1bc499fc4c2aa4f5d08fcf690ff5)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:39.928094 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6701be98-41ca-4b8d-a1a2-ccabb855377b admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:39.941608 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db99a378-9097-47c9-9ae3-6e20eb41c94f admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:39.943974 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-db99a378-9097-47c9-9ae3-6e20eb41c94f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:39.964948 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db99a378-9097-47c9-9ae3-6e20eb41c94f admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:39.969954 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38248fc-7151-4b88-805e-84fe4d5414b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:39.970606 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38248fc-7151-4b88-805e-84fe4d5414b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:39.970865 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38248fc-7151-4b88-805e-84fe4d5414b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:39.971174 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38248fc-7151-4b88-805e-84fe4d5414b3 admin admin] PATH_INFO: `/v3/users/b2e9c8c3fae24896a8b59abedb2e45a9` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:39.972428 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38248fc-7151-4b88-805e-84fe4d5414b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2e9c8c3fae24896a8b59abedb2e45a9)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:39.974341 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38248fc-7151-4b88-805e-84fe4d5414b3 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:40.123948 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-6701be98-41ca-4b8d-a1a2-ccabb855377b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '951a1bc499fc4c2aa4f5d08fcf690ff5'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:40.146295 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-b38248fc-7151-4b88-805e-84fe4d5414b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2e9c8c3fae24896a8b59abedb2e45a9'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:40.158682 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-b38248fc-7151-4b88-805e-84fe4d5414b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2e9c8c3fae24896a8b59abedb2e45a9'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:40.159408 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-b38248fc-7151-4b88-805e-84fe4d5414b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2e9c8c3fae24896a8b59abedb2e45a9'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:40.207416 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-6701be98-41ca-4b8d-a1a2-ccabb855377b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '951a1bc499fc4c2aa4f5d08fcf690ff5'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:40.208122 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-6701be98-41ca-4b8d-a1a2-ccabb855377b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '951a1bc499fc4c2aa4f5d08fcf690ff5'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:40.208740 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-6701be98-41ca-4b8d-a1a2-ccabb855377b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '951a1bc499fc4c2aa4f5d08fcf690ff5'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:40.219484 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-6701be98-41ca-4b8d-a1a2-ccabb855377b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '951a1bc499fc4c2aa4f5d08fcf690ff5'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:40.243337 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 994/1960] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:13:39 2022] DELETE /identity/v3/users/951a1bc499fc4c2aa4f5d08fcf690ff5 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:13:40.349782 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-b38248fc-7151-4b88-805e-84fe4d5414b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2e9c8c3fae24896a8b59abedb2e45a9'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:40.350487 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-b38248fc-7151-4b88-805e-84fe4d5414b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2e9c8c3fae24896a8b59abedb2e45a9'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:40.369729 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 967/1961] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:13:39 2022] DELETE /identity/v3/users/b2e9c8c3fae24896a8b59abedb2e45a9 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:13:40.379377 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6c86eb-28fb-47f6-974d-033d192d110b None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:40.379732 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6c86eb-28fb-47f6-974d-033d192d110b None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:40.380069 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6c86eb-28fb-47f6-974d-033d192d110b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:40.401957 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-0a6c86eb-28fb-47f6-974d-033d192d110b None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:13:40.403218 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-0a6c86eb-28fb-47f6-974d-033d192d110b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:40.416653 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:13:40.466468 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 995/1962] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:13:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:13:40.476699 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38248fc-7151-4b88-805e-84fe4d5414b3 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:40.488538 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-b38248fc-7151-4b88-805e-84fe4d5414b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:40.631307 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38248fc-7151-4b88-805e-84fe4d5414b3 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:40.636552 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df66feaf-4112-4d90-96d4-b93e5e532201 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:40.637186 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df66feaf-4112-4d90-96d4-b93e5e532201 service neutron] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:40.637530 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df66feaf-4112-4d90-96d4-b93e5e532201 service neutron] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:40.638068 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df66feaf-4112-4d90-96d4-b93e5e532201 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:40.653386 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df66feaf-4112-4d90-96d4-b93e5e532201 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:40.655099 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df66feaf-4112-4d90-96d4-b93e5e532201 service neutron] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:40.682144 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 968/1963] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:13:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:13:41.292843 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6c86eb-28fb-47f6-974d-033d192d110b None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:41.295210 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-0a6c86eb-28fb-47f6-974d-033d192d110b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:41.361685 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6c86eb-28fb-47f6-974d-033d192d110b None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:41.365702 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2e564b-28fe-4c79-9021-894f1877a865 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:41.366319 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2e564b-28fe-4c79-9021-894f1877a865 service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:41.366595 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2e564b-28fe-4c79-9021-894f1877a865 service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:41.366869 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2e564b-28fe-4c79-9021-894f1877a865 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:41.369490 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-de2e564b-28fe-4c79-9021-894f1877a865 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:41.437069 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2e564b-28fe-4c79-9021-894f1877a865 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:41.437069 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2e564b-28fe-4c79-9021-894f1877a865 service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:41.438483 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df66feaf-4112-4d90-96d4-b93e5e532201 service neutron] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:41.440210 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-df66feaf-4112-4d90-96d4-b93e5e532201 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:41.457173 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 996/1964] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:13:41 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:13:41.467510 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df66feaf-4112-4d90-96d4-b93e5e532201 service neutron] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:41.475142 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb473a4-2904-4c41-894e-9170947461e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:41.475930 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb473a4-2904-4c41-894e-9170947461e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:41.476220 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb473a4-2904-4c41-894e-9170947461e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:41.476498 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb473a4-2904-4c41-894e-9170947461e5 admin admin] PATH_INFO: `/v3/projects/1fd1c05e11f749558bab5ec8a0608220` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:41.477863 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb473a4-2904-4c41-894e-9170947461e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fd1c05e11f749558bab5ec8a0608220)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:41.479862 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb473a4-2904-4c41-894e-9170947461e5 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:41.646099 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-4fb473a4-2904-4c41-894e-9170947461e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fd1c05e11f749558bab5ec8a0608220', 'request_id': 'req-4fb473a4-2904-4c41-894e-9170947461e5'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:41.735620 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-4fb473a4-2904-4c41-894e-9170947461e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fd1c05e11f749558bab5ec8a0608220', 'request_id': 'req-4fb473a4-2904-4c41-894e-9170947461e5'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:41.745231 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:13:41.745945 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-4fb473a4-2904-4c41-894e-9170947461e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fd1c05e11f749558bab5ec8a0608220', 'request_id': 'req-4fb473a4-2904-4c41-894e-9170947461e5'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:41.748965 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2e564b-28fe-4c79-9021-894f1877a865 service cinder] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:41.751473 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-de2e564b-28fe-4c79-9021-894f1877a865 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:41.769283 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-4fb473a4-2904-4c41-894e-9170947461e5 admin admin] The token cache is being invalidate because project 1fd1c05e11f749558bab5ec8a0608220 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81650) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:13:41.769640 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-4fb473a4-2904-4c41-894e-9170947461e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:41.770479 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-4fb473a4-2904-4c41-894e-9170947461e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:41.773292 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 969/1965] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:13:41 2022] DELETE /identity/v3/projects/1fd1c05e11f749558bab5ec8a0608220 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:13:41.776658 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2e564b-28fe-4c79-9021-894f1877a865 service cinder] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:41.778338 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec7d48-48bf-45c8-abe0-9cad9e3a4aa7 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:41.778635 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec7d48-48bf-45c8-abe0-9cad9e3a4aa7 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:41.779092 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec7d48-48bf-45c8-abe0-9cad9e3a4aa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:41.822782 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-98ec7d48-48bf-45c8-abe0-9cad9e3a4aa7 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:13:41.824039 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-98ec7d48-48bf-45c8-abe0-9cad9e3a4aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:41.869465 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eae46d-2ea1-4381-a0e3-3d0880478522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:41.870108 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eae46d-2ea1-4381-a0e3-3d0880478522 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:41.870378 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eae46d-2ea1-4381-a0e3-3d0880478522 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:41.870698 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eae46d-2ea1-4381-a0e3-3d0880478522 admin admin] PATH_INFO: `/v3/projects/a558e5dae59c4cd2b0dcd5d0990afb18` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:41.871997 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eae46d-2ea1-4381-a0e3-3d0880478522 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a558e5dae59c4cd2b0dcd5d0990afb18)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:41.873937 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eae46d-2ea1-4381-a0e3-3d0880478522 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:41.893481 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:13:41.934854 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 970/1966] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:13:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:13:41.936924 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49207ba2-fc51-47f8-af2c-7a5ed0a61d8e None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:41.936924 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49207ba2-fc51-47f8-af2c-7a5ed0a61d8e None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:41.937275 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49207ba2-fc51-47f8-af2c-7a5ed0a61d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:41.958240 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-49207ba2-fc51-47f8-af2c-7a5ed0a61d8e None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:13:41.959495 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-49207ba2-fc51-47f8-af2c-7a5ed0a61d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:41.967752 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 971/1967] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:13:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:13:41.972033 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49207ba2-fc51-47f8-af2c-7a5ed0a61d8e None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:41.981491 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49207ba2-fc51-47f8-af2c-7a5ed0a61d8e None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:41.986522 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c370b0-dc11-4298-87a2-c76cd6f24760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:41.987198 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c370b0-dc11-4298-87a2-c76cd6f24760 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:41.987488 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c370b0-dc11-4298-87a2-c76cd6f24760 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:41.987790 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c370b0-dc11-4298-87a2-c76cd6f24760 admin admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:41.988454 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c370b0-dc11-4298-87a2-c76cd6f24760 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:13:41.988749 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c370b0-dc11-4298-87a2-c76cd6f24760 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:41.990372 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c370b0-dc11-4298-87a2-c76cd6f24760 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:42.003704 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 972/1968] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:13:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 17:13:42.008170 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c370b0-dc11-4298-87a2-c76cd6f24760 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:42.018539 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c370b0-dc11-4298-87a2-c76cd6f24760 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:42.030052 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb7b457-7d3a-48de-be24-70cf74b99975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:42.030673 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb7b457-7d3a-48de-be24-70cf74b99975 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:42.031170 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb7b457-7d3a-48de-be24-70cf74b99975 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:42.031285 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb7b457-7d3a-48de-be24-70cf74b99975 admin admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:42.032022 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb7b457-7d3a-48de-be24-70cf74b99975 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:42.034782 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb7b457-7d3a-48de-be24-70cf74b99975 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:42.048978 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-38eae46d-2ea1-4381-a0e3-3d0880478522 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a558e5dae59c4cd2b0dcd5d0990afb18', 'request_id': 'req-38eae46d-2ea1-4381-a0e3-3d0880478522'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:42.056157 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-38eae46d-2ea1-4381-a0e3-3d0880478522 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a558e5dae59c4cd2b0dcd5d0990afb18', 'request_id': 'req-38eae46d-2ea1-4381-a0e3-3d0880478522'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:42.133037 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-38eae46d-2ea1-4381-a0e3-3d0880478522 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a558e5dae59c4cd2b0dcd5d0990afb18', 'request_id': 'req-38eae46d-2ea1-4381-a0e3-3d0880478522'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:42.142794 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 973/1969] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:13:42 2022] POST /identity/v3/projects => generated 385 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 17:13:42.145359 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-38eae46d-2ea1-4381-a0e3-3d0880478522 admin admin] The token cache is being invalidate because project a558e5dae59c4cd2b0dcd5d0990afb18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:13:42.145682 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-38eae46d-2ea1-4381-a0e3-3d0880478522 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:42.146293 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-38eae46d-2ea1-4381-a0e3-3d0880478522 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:42.146863 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb7b457-7d3a-48de-be24-70cf74b99975 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:42.151348 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 997/1970] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:13:41 2022] DELETE /identity/v3/projects/a558e5dae59c4cd2b0dcd5d0990afb18 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:13:42.151492 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-bcb7b457-7d3a-48de-be24-70cf74b99975 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:42.152329 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eae46d-2ea1-4381-a0e3-3d0880478522 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:42.153647 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-38eae46d-2ea1-4381-a0e3-3d0880478522 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:42.226235 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb7b457-7d3a-48de-be24-70cf74b99975 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:42.232664 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8afb474-9497-4104-a4ba-c10942fd0207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:42.233403 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8afb474-9497-4104-a4ba-c10942fd0207 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:42.233788 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8afb474-9497-4104-a4ba-c10942fd0207 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:42.234164 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8afb474-9497-4104-a4ba-c10942fd0207 admin admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:42.235084 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8afb474-9497-4104-a4ba-c10942fd0207 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:42.246984 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8afb474-9497-4104-a4ba-c10942fd0207 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:42.257708 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eae46d-2ea1-4381-a0e3-3d0880478522 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:42.262568 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3f4e2b-1e98-4226-bad8-9049780c6652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:42.263357 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f4e2b-1e98-4226-bad8-9049780c6652 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:42.263649 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f4e2b-1e98-4226-bad8-9049780c6652 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:42.263921 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f4e2b-1e98-4226-bad8-9049780c6652 admin admin] PATH_INFO: `/v3/users/01246c1b553047a69dfbbd4207728451` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:42.265061 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3f4e2b-1e98-4226-bad8-9049780c6652 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01246c1b553047a69dfbbd4207728451)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:42.267187 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3f4e2b-1e98-4226-bad8-9049780c6652 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:42.362524 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 974/1971] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:13:42 2022] POST /identity/v3/users => generated 293 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:13:42.363487 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8afb474-9497-4104-a4ba-c10942fd0207 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:42.365233 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-b8afb474-9497-4104-a4ba-c10942fd0207 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:42.377696 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8afb474-9497-4104-a4ba-c10942fd0207 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:42.382942 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1da46b7-4077-49b0-8fd8-730c400a34e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:42.383561 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1da46b7-4077-49b0-8fd8-730c400a34e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:42.383919 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1da46b7-4077-49b0-8fd8-730c400a34e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:42.384197 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1da46b7-4077-49b0-8fd8-730c400a34e3 admin admin] PATH_INFO: `/v3/users/afcaa4c9a2774a7388a6fcf90ba58449` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:42.385351 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1da46b7-4077-49b0-8fd8-730c400a34e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afcaa4c9a2774a7388a6fcf90ba58449)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:42.387340 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1da46b7-4077-49b0-8fd8-730c400a34e3 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:42.454979 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-0f3f4e2b-1e98-4226-bad8-9049780c6652 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01246c1b553047a69dfbbd4207728451'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:42.584131 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-0f3f4e2b-1e98-4226-bad8-9049780c6652 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01246c1b553047a69dfbbd4207728451'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:42.584866 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-0f3f4e2b-1e98-4226-bad8-9049780c6652 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01246c1b553047a69dfbbd4207728451'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:42.585563 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-0f3f4e2b-1e98-4226-bad8-9049780c6652 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01246c1b553047a69dfbbd4207728451'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:42.594280 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-0f3f4e2b-1e98-4226-bad8-9049780c6652 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01246c1b553047a69dfbbd4207728451'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:42.620630 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 998/1972] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:13:42 2022] DELETE /identity/v3/users/01246c1b553047a69dfbbd4207728451 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:13:42.622458 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3f4e2b-1e98-4226-bad8-9049780c6652 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:42.624304 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-0f3f4e2b-1e98-4226-bad8-9049780c6652 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:42.646707 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-f1da46b7-4077-49b0-8fd8-730c400a34e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afcaa4c9a2774a7388a6fcf90ba58449'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:42.664034 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-f1da46b7-4077-49b0-8fd8-730c400a34e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afcaa4c9a2774a7388a6fcf90ba58449'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:42.664961 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-f1da46b7-4077-49b0-8fd8-730c400a34e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afcaa4c9a2774a7388a6fcf90ba58449'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:42.666523 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:13:42.724291 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3f4e2b-1e98-4226-bad8-9049780c6652 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:42.732200 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0d3d17-4784-42da-a916-97687e92d134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:42.732200 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d3d17-4784-42da-a916-97687e92d134 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:42.732200 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d3d17-4784-42da-a916-97687e92d134 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:42.732200 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d3d17-4784-42da-a916-97687e92d134 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:42.732550 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0d3d17-4784-42da-a916-97687e92d134 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:42.733705 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0d3d17-4784-42da-a916-97687e92d134 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:42.744452 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 999/1973] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:13:42 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:13:42.748413 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0d3d17-4784-42da-a916-97687e92d134 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:42.758625 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0d3d17-4784-42da-a916-97687e92d134 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:42.763785 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f34e5f-54c9-4db7-998d-64c7f2cd0091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:42.764396 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f34e5f-54c9-4db7-998d-64c7f2cd0091 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:42.764675 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f34e5f-54c9-4db7-998d-64c7f2cd0091 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:42.764944 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f34e5f-54c9-4db7-998d-64c7f2cd0091 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:42.765576 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f34e5f-54c9-4db7-998d-64c7f2cd0091 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:42.767510 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f34e5f-54c9-4db7-998d-64c7f2cd0091 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:42.770844 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-f1da46b7-4077-49b0-8fd8-730c400a34e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afcaa4c9a2774a7388a6fcf90ba58449'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:42.771579 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-f1da46b7-4077-49b0-8fd8-730c400a34e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afcaa4c9a2774a7388a6fcf90ba58449'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:42.779954 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1000/1974] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:13:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 17:13:42.785663 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 975/1975] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:13:42 2022] DELETE /identity/v3/users/afcaa4c9a2774a7388a6fcf90ba58449 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:13:42.787014 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f34e5f-54c9-4db7-998d-64c7f2cd0091 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:42.790187 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cd4bd8-ba2e-4cfe-9d3a-06f80e7fe0c3 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:42.790485 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cd4bd8-ba2e-4cfe-9d3a-06f80e7fe0c3 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:42.790729 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-22f34e5f-54c9-4db7-998d-64c7f2cd0091 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:42.790832 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cd4bd8-ba2e-4cfe-9d3a-06f80e7fe0c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:42.803595 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f34e5f-54c9-4db7-998d-64c7f2cd0091 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:42.810443 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-c9cd4bd8-ba2e-4cfe-9d3a-06f80e7fe0c3 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:13:42.811892 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-c9cd4bd8-ba2e-4cfe-9d3a-06f80e7fe0c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:42.874491 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dc7be4-5fda-4a74-bd04-0c3b58137d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:42.875199 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc7be4-5fda-4a74-bd04-0c3b58137d3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:42.880145 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc7be4-5fda-4a74-bd04-0c3b58137d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:42.880145 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc7be4-5fda-4a74-bd04-0c3b58137d3b admin admin] PATH_INFO: `/v3/projects/5af8b5d29b074a9daacecf15e7c17fa0/users/0b262221a873438cb1600d83026b5dbd/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:42.880566 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 976/1976] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:13:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:13:42.897505 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dc7be4-5fda-4a74-bd04-0c3b58137d3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5af8b5d29b074a9daacecf15e7c17fa0, user_id=0b262221a873438cb1600d83026b5dbd, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:42.898015 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cd4bd8-ba2e-4cfe-9d3a-06f80e7fe0c3 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:42.900993 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-c9cd4bd8-ba2e-4cfe-9d3a-06f80e7fe0c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:42.901423 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dc7be4-5fda-4a74-bd04-0c3b58137d3b admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:42.959581 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cd4bd8-ba2e-4cfe-9d3a-06f80e7fe0c3 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:42.963449 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f5de24-60e1-4830-a5a1-a67b2a515324 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:42.964111 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f5de24-60e1-4830-a5a1-a67b2a515324 service neutron] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:42.964383 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f5de24-60e1-4830-a5a1-a67b2a515324 service neutron] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:42.964660 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f5de24-60e1-4830-a5a1-a67b2a515324 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:42.975019 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f5de24-60e1-4830-a5a1-a67b2a515324 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:42.976520 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f5de24-60e1-4830-a5a1-a67b2a515324 service neutron] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:42.988934 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 977/1977] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:13:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:13:43.016996 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1001/1978] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:13:42 2022] PUT /identity/v3/projects/5af8b5d29b074a9daacecf15e7c17fa0/users/0b262221a873438cb1600d83026b5dbd/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:13:43.027496 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ffbc2f-1203-462d-a52b-05ce78996c2b None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:43.027701 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ffbc2f-1203-462d-a52b-05ce78996c2b None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:43.028008 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ffbc2f-1203-462d-a52b-05ce78996c2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:43.075789 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-49ffbc2f-1203-462d-a52b-05ce78996c2b None None] MFA Rules not processed for user `0b262221a873438cb1600d83026b5dbd`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:13:43.077210 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-49ffbc2f-1203-462d-a52b-05ce78996c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:43.119173 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dc7be4-5fda-4a74-bd04-0c3b58137d3b admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:43.121014 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-d0dc7be4-5fda-4a74-bd04-0c3b58137d3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:43.187425 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 978/1979] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:13:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:13:43.188396 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ffbc2f-1203-462d-a52b-05ce78996c2b None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:43.201082 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-49ffbc2f-1203-462d-a52b-05ce78996c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:43.231408 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dc7be4-5fda-4a74-bd04-0c3b58137d3b admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:43.236431 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2fdb2b-2ffb-42d8-8341-f2a42906ed0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:43.237170 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2fdb2b-2ffb-42d8-8341-f2a42906ed0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:43.237522 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2fdb2b-2ffb-42d8-8341-f2a42906ed0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:43.237827 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2fdb2b-2ffb-42d8-8341-f2a42906ed0e admin admin] PATH_INFO: `/v3/projects/274617e664c04b7f8d72d74c09d14593` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:43.239152 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2fdb2b-2ffb-42d8-8341-f2a42906ed0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=274617e664c04b7f8d72d74c09d14593)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:43.241852 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2fdb2b-2ffb-42d8-8341-f2a42906ed0e admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:43.269199 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ffbc2f-1203-462d-a52b-05ce78996c2b None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:43.272894 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b57cd8-53fb-4f8d-abb0-70fd20126551 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:43.273509 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b57cd8-53fb-4f8d-abb0-70fd20126551 service cinder] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:43.274937 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b57cd8-53fb-4f8d-abb0-70fd20126551 service cinder] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:43.274937 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b57cd8-53fb-4f8d-abb0-70fd20126551 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:43.279100 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-63b57cd8-53fb-4f8d-abb0-70fd20126551 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:43.345349 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b57cd8-53fb-4f8d-abb0-70fd20126551 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:43.346840 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b57cd8-53fb-4f8d-abb0-70fd20126551 service cinder] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:43.348736 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-aa2fdb2b-2ffb-42d8-8341-f2a42906ed0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '274617e664c04b7f8d72d74c09d14593', 'request_id': 'req-aa2fdb2b-2ffb-42d8-8341-f2a42906ed0e'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:43.356260 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-aa2fdb2b-2ffb-42d8-8341-f2a42906ed0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '274617e664c04b7f8d72d74c09d14593', 'request_id': 'req-aa2fdb2b-2ffb-42d8-8341-f2a42906ed0e'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:43.365952 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 979/1980] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:13:43 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:13:43.366829 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b57cd8-53fb-4f8d-abb0-70fd20126551 service cinder] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:43.380776 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b57cd8-53fb-4f8d-abb0-70fd20126551 service cinder] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:43.428315 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-aa2fdb2b-2ffb-42d8-8341-f2a42906ed0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '274617e664c04b7f8d72d74c09d14593', 'request_id': 'req-aa2fdb2b-2ffb-42d8-8341-f2a42906ed0e'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:43.447036 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-aa2fdb2b-2ffb-42d8-8341-f2a42906ed0e admin admin] The token cache is being invalidate because project 274617e664c04b7f8d72d74c09d14593 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:13:43.447380 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-aa2fdb2b-2ffb-42d8-8341-f2a42906ed0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:43.449132 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-aa2fdb2b-2ffb-42d8-8341-f2a42906ed0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:43.452019 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1002/1981] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:13:43 2022] DELETE /identity/v3/projects/274617e664c04b7f8d72d74c09d14593 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:13:43.453412 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:13:43.465158 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde55b5a-a868-400f-88d0-fb6ab33ad4bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:43.466954 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde55b5a-a868-400f-88d0-fb6ab33ad4bf admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:43.467522 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde55b5a-a868-400f-88d0-fb6ab33ad4bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:43.468404 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde55b5a-a868-400f-88d0-fb6ab33ad4bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:43.469211 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde55b5a-a868-400f-88d0-fb6ab33ad4bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:43.471557 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde55b5a-a868-400f-88d0-fb6ab33ad4bf admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:43.472506 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434ec3c8-9d06-4941-af9a-0e3926a91466 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:43.475101 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434ec3c8-9d06-4941-af9a-0e3926a91466 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:43.475555 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434ec3c8-9d06-4941-af9a-0e3926a91466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:43.513016 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-434ec3c8-9d06-4941-af9a-0e3926a91466 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:13:43.514255 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-434ec3c8-9d06-4941-af9a-0e3926a91466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:43.522504 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1003/1982] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:13:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:13:43.531029 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0a0e53-8fd4-4aec-aa53-5a81dde23964 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:43.531938 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0a0e53-8fd4-4aec-aa53-5a81dde23964 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:43.532216 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0a0e53-8fd4-4aec-aa53-5a81dde23964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:43.553471 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 980/1983] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:13:43 2022] POST /identity/v3/projects => generated 385 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:13:43.553749 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-5f0a0e53-8fd4-4aec-aa53-5a81dde23964 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:13:43.554995 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-5f0a0e53-8fd4-4aec-aa53-5a81dde23964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:43.557624 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde55b5a-a868-400f-88d0-fb6ab33ad4bf admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:43.559567 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-fde55b5a-a868-400f-88d0-fb6ab33ad4bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:43.643143 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1004/1984] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:13:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:13:43.643143 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0a0e53-8fd4-4aec-aa53-5a81dde23964 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:43.644499 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-5f0a0e53-8fd4-4aec-aa53-5a81dde23964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:43.673731 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0a0e53-8fd4-4aec-aa53-5a81dde23964 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:43.683178 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f64404-816d-49b8-834c-fc96a7ca496b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:43.683923 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f64404-816d-49b8-834c-fc96a7ca496b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:43.684035 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde55b5a-a868-400f-88d0-fb6ab33ad4bf admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:43.684476 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f64404-816d-49b8-834c-fc96a7ca496b admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:43.684476 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f64404-816d-49b8-834c-fc96a7ca496b admin admin] PATH_INFO: `/v3/projects/7053619defb64841aa8e0a807c8f7367` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:43.685622 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f64404-816d-49b8-834c-fc96a7ca496b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7053619defb64841aa8e0a807c8f7367)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:43.687675 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f64404-816d-49b8-834c-fc96a7ca496b admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:43.689071 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d060a37f-335f-4272-ae4f-1d0f9d4fb78c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:43.691566 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d060a37f-335f-4272-ae4f-1d0f9d4fb78c admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:43.691566 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d060a37f-335f-4272-ae4f-1d0f9d4fb78c admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:43.691566 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d060a37f-335f-4272-ae4f-1d0f9d4fb78c admin admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:43.691566 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d060a37f-335f-4272-ae4f-1d0f9d4fb78c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:43.692836 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d060a37f-335f-4272-ae4f-1d0f9d4fb78c admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:43.801612 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 981/1985] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:13:43 2022] POST /identity/v3/users => generated 299 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:13:43.802706 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d060a37f-335f-4272-ae4f-1d0f9d4fb78c admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:43.813867 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d060a37f-335f-4272-ae4f-1d0f9d4fb78c admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:43.818541 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beb5308-a497-4443-a27d-c9a86923a155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:43.819244 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb5308-a497-4443-a27d-c9a86923a155 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:43.819534 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb5308-a497-4443-a27d-c9a86923a155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:43.819814 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb5308-a497-4443-a27d-c9a86923a155 admin admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:43.821198 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beb5308-a497-4443-a27d-c9a86923a155 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:13:43.821527 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beb5308-a497-4443-a27d-c9a86923a155 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:43.823419 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beb5308-a497-4443-a27d-c9a86923a155 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:43.839509 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 982/1986] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:13:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 17:13:43.840814 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beb5308-a497-4443-a27d-c9a86923a155 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:43.853836 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beb5308-a497-4443-a27d-c9a86923a155 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:43.865109 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e7f64404-816d-49b8-834c-fc96a7ca496b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7053619defb64841aa8e0a807c8f7367', 'request_id': 'req-e7f64404-816d-49b8-834c-fc96a7ca496b'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:43.872713 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e7f64404-816d-49b8-834c-fc96a7ca496b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7053619defb64841aa8e0a807c8f7367', 'request_id': 'req-e7f64404-816d-49b8-834c-fc96a7ca496b'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:43.915117 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec88d30-bb78-4be8-8c43-96319a2eb38e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:43.915723 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec88d30-bb78-4be8-8c43-96319a2eb38e admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:43.916006 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec88d30-bb78-4be8-8c43-96319a2eb38e admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:43.916278 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec88d30-bb78-4be8-8c43-96319a2eb38e admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:43.916904 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec88d30-bb78-4be8-8c43-96319a2eb38e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:43.918775 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec88d30-bb78-4be8-8c43-96319a2eb38e admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:43.928467 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 983/1987] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:13:43 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:13:43.929355 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec88d30-bb78-4be8-8c43-96319a2eb38e admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:43.939373 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec88d30-bb78-4be8-8c43-96319a2eb38e admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:43.944229 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7987622-49a0-4cba-a653-3d7a4969cf2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:43.944833 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7987622-49a0-4cba-a653-3d7a4969cf2b admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:43.945114 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7987622-49a0-4cba-a653-3d7a4969cf2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:43.945385 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7987622-49a0-4cba-a653-3d7a4969cf2b admin admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:43.946852 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7987622-49a0-4cba-a653-3d7a4969cf2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:43.948939 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7987622-49a0-4cba-a653-3d7a4969cf2b admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:43.955252 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e7f64404-816d-49b8-834c-fc96a7ca496b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7053619defb64841aa8e0a807c8f7367', 'request_id': 'req-e7f64404-816d-49b8-834c-fc96a7ca496b'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:43.966655 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e7f64404-816d-49b8-834c-fc96a7ca496b admin admin] The token cache is being invalidate because project 7053619defb64841aa8e0a807c8f7367 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:13:43.966982 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e7f64404-816d-49b8-834c-fc96a7ca496b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:43.967654 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e7f64404-816d-49b8-834c-fc96a7ca496b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:43.969181 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1005/1988] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:13:43 2022] DELETE /identity/v3/projects/7053619defb64841aa8e0a807c8f7367 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:13:43.969990 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f64404-816d-49b8-834c-fc96a7ca496b admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:43.971395 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-e7f64404-816d-49b8-834c-fc96a7ca496b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:43.985331 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f64404-816d-49b8-834c-fc96a7ca496b admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:43.989919 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c4cd17-d14a-4d6d-b0cb-42cc48bb265e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:43.990561 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c4cd17-d14a-4d6d-b0cb-42cc48bb265e admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:43.994149 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c4cd17-d14a-4d6d-b0cb-42cc48bb265e admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:43.994149 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c4cd17-d14a-4d6d-b0cb-42cc48bb265e admin admin] PATH_INFO: `/v3/projects/8ae4c5ca8064460c9859957e1b05d58b/users/52386e8c2c4e46a9823f419d5e19ed4c/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:44.006197 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c4cd17-d14a-4d6d-b0cb-42cc48bb265e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ae4c5ca8064460c9859957e1b05d58b, user_id=52386e8c2c4e46a9823f419d5e19ed4c, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:44.010055 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c4cd17-d14a-4d6d-b0cb-42cc48bb265e admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:44.037001 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 984/1989] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:13:43 2022] POST /identity/v3/projects => generated 407 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:13:44.038469 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbfb368-ed9b-4a87-b69b-5724b94d3c26 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:44.038752 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbfb368-ed9b-4a87-b69b-5724b94d3c26 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:44.039057 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbfb368-ed9b-4a87-b69b-5724b94d3c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:44.059386 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-1cbfb368-ed9b-4a87-b69b-5724b94d3c26 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:13:44.060569 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-1cbfb368-ed9b-4a87-b69b-5724b94d3c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:44.081841 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1006/1990] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:13:43 2022] PUT /identity/v3/projects/8ae4c5ca8064460c9859957e1b05d58b/users/52386e8c2c4e46a9823f419d5e19ed4c/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:13:44.082992 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c4cd17-d14a-4d6d-b0cb-42cc48bb265e admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:44.084608 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-38c4cd17-d14a-4d6d-b0cb-42cc48bb265e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:44.122869 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 985/1991] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:13:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:13:44.123902 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbfb368-ed9b-4a87-b69b-5724b94d3c26 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:44.133424 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbfb368-ed9b-4a87-b69b-5724b94d3c26 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:44.155782 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c4cd17-d14a-4d6d-b0cb-42cc48bb265e admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:44.160819 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cbb688-ade2-492a-979c-1a8fe01bfe2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:44.161468 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cbb688-ade2-492a-979c-1a8fe01bfe2b admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:44.161751 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cbb688-ade2-492a-979c-1a8fe01bfe2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:44.162041 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cbb688-ade2-492a-979c-1a8fe01bfe2b admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:44.162646 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cbb688-ade2-492a-979c-1a8fe01bfe2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:44.165342 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cbb688-ade2-492a-979c-1a8fe01bfe2b admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:44.192000 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c99963f-9bf3-4308-ba1f-d245a9e3f4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:44.192616 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c99963f-9bf3-4308-ba1f-d245a9e3f4ca admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:44.192898 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c99963f-9bf3-4308-ba1f-d245a9e3f4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:44.193169 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c99963f-9bf3-4308-ba1f-d245a9e3f4ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:44.193810 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c99963f-9bf3-4308-ba1f-d245a9e3f4ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:44.195461 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c99963f-9bf3-4308-ba1f-d245a9e3f4ca admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:44.205127 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 986/1992] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:13:44.206508 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87597d19-872b-4610-b4c2-b5783ce06c2f None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:44.206800 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87597d19-872b-4610-b4c2-b5783ce06c2f None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:44.207109 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87597d19-872b-4610-b4c2-b5783ce06c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:44.232319 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-87597d19-872b-4610-b4c2-b5783ce06c2f None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:13:44.233592 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-87597d19-872b-4610-b4c2-b5783ce06c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:44.237015 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:13:44.242497 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 987/1993] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:13:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:13:44.243840 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87597d19-872b-4610-b4c2-b5783ce06c2f None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:44.253939 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87597d19-872b-4610-b4c2-b5783ce06c2f None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:44.259802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e94988f-a949-4ab3-9b57-ea6214b482dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:44.260503 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e94988f-a949-4ab3-9b57-ea6214b482dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:44.260810 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e94988f-a949-4ab3-9b57-ea6214b482dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:44.261063 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e94988f-a949-4ab3-9b57-ea6214b482dd admin admin] PATH_INFO: `/v3/projects/8ae4c5ca8064460c9859957e1b05d58b/users/52386e8c2c4e46a9823f419d5e19ed4c/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:44.263390 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e94988f-a949-4ab3-9b57-ea6214b482dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ae4c5ca8064460c9859957e1b05d58b, user_id=52386e8c2c4e46a9823f419d5e19ed4c, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:44.266122 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e94988f-a949-4ab3-9b57-ea6214b482dd admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:44.277749 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 988/1994] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:13:44 2022] PUT /identity/v3/projects/8ae4c5ca8064460c9859957e1b05d58b/users/52386e8c2c4e46a9823f419d5e19ed4c/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:13:44.278804 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e94988f-a949-4ab3-9b57-ea6214b482dd admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:44.295385 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1007/1995] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:13:44 2022] POST /identity/v3/users => generated 304 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:13:44.296732 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfc74d3-bf89-454c-a7d6-d7f096dbcd1e None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:44.297023 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfc74d3-bf89-454c-a7d6-d7f096dbcd1e None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:44.297357 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfc74d3-bf89-454c-a7d6-d7f096dbcd1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:44.327157 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-acfc74d3-bf89-454c-a7d6-d7f096dbcd1e None None] MFA Rules not processed for user `52386e8c2c4e46a9823f419d5e19ed4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:13:44.328400 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-acfc74d3-bf89-454c-a7d6-d7f096dbcd1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:44.352941 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e94988f-a949-4ab3-9b57-ea6214b482dd admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:44.359076 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcfef9f-6d04-49af-90a1-ff762ef193aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:44.359076 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcfef9f-6d04-49af-90a1-ff762ef193aa admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:44.359397 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcfef9f-6d04-49af-90a1-ff762ef193aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:44.359670 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcfef9f-6d04-49af-90a1-ff762ef193aa admin admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:44.360324 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcfef9f-6d04-49af-90a1-ff762ef193aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:13:44.360631 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcfef9f-6d04-49af-90a1-ff762ef193aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:44.362257 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcfef9f-6d04-49af-90a1-ff762ef193aa admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:44.376499 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 989/1996] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:13:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 17:13:44.379907 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcfef9f-6d04-49af-90a1-ff762ef193aa admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:44.394103 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcfef9f-6d04-49af-90a1-ff762ef193aa admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:44.404304 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca65260-5b6d-4377-a3d3-6adb00bbdd28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:44.404928 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca65260-5b6d-4377-a3d3-6adb00bbdd28 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:44.405225 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca65260-5b6d-4377-a3d3-6adb00bbdd28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:44.405553 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca65260-5b6d-4377-a3d3-6adb00bbdd28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:44.406240 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca65260-5b6d-4377-a3d3-6adb00bbdd28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:44.407946 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca65260-5b6d-4377-a3d3-6adb00bbdd28 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:44.413091 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1008/1997] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:13:44 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:13:44.413960 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfc74d3-bf89-454c-a7d6-d7f096dbcd1e None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:44.431117 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfc74d3-bf89-454c-a7d6-d7f096dbcd1e None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:44.432621 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c24b01b-4074-4647-852f-8da0cd745cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:44.433231 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c24b01b-4074-4647-852f-8da0cd745cd8 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:44.433508 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c24b01b-4074-4647-852f-8da0cd745cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:44.434205 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c24b01b-4074-4647-852f-8da0cd745cd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:44.434581 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c24b01b-4074-4647-852f-8da0cd745cd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:44.435583 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 990/1998] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:13:44.436532 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca65260-5b6d-4377-a3d3-6adb00bbdd28 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:44.436658 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c24b01b-4074-4647-852f-8da0cd745cd8 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:44.438691 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-3ca65260-5b6d-4377-a3d3-6adb00bbdd28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:44.498764 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca65260-5b6d-4377-a3d3-6adb00bbdd28 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:44.502531 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ec97eb-09d5-4dce-95c0-f57a2fa77b9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:44.503178 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ec97eb-09d5-4dce-95c0-f57a2fa77b9a service cinder] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:44.503465 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ec97eb-09d5-4dce-95c0-f57a2fa77b9a service cinder] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:44.503740 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ec97eb-09d5-4dce-95c0-f57a2fa77b9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:44.513723 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ec97eb-09d5-4dce-95c0-f57a2fa77b9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:44.515535 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ec97eb-09d5-4dce-95c0-f57a2fa77b9a service cinder] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:44.526327 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 991/1999] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:13:44 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:13:44.527070 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ec97eb-09d5-4dce-95c0-f57a2fa77b9a service cinder] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:44.540459 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ec97eb-09d5-4dce-95c0-f57a2fa77b9a service cinder] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:44.564081 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b765b49-ead5-479c-a0f4-1327b5873dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:44.565081 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b765b49-ead5-479c-a0f4-1327b5873dcd admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:44.565372 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b765b49-ead5-479c-a0f4-1327b5873dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:44.565648 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b765b49-ead5-479c-a0f4-1327b5873dcd admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:44.566292 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b765b49-ead5-479c-a0f4-1327b5873dcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:44.575903 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b765b49-ead5-479c-a0f4-1327b5873dcd admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:44.582402 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1009/2000] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:13:44 2022] POST /identity/v3/projects => generated 369 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:13:44.583947 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c24b01b-4074-4647-852f-8da0cd745cd8 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:44.611724 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 992/2001] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:13:44.617832 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b765b49-ead5-479c-a0f4-1327b5873dcd admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:44.653002 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b765b49-ead5-479c-a0f4-1327b5873dcd admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:44.677958 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c24b01b-4074-4647-852f-8da0cd745cd8 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:44.703642 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a2260c-9d1d-411f-b80b-a63fbe1bdfe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:44.705009 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a2260c-9d1d-411f-b80b-a63fbe1bdfe3 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:44.705305 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a2260c-9d1d-411f-b80b-a63fbe1bdfe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:44.705561 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a2260c-9d1d-411f-b80b-a63fbe1bdfe3 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:44.708272 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a2260c-9d1d-411f-b80b-a63fbe1bdfe3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:44.715308 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a2260c-9d1d-411f-b80b-a63fbe1bdfe3 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:44.743807 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39162286-7d8c-4831-9bd3-902494283495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:44.744540 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39162286-7d8c-4831-9bd3-902494283495 admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:44.744830 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39162286-7d8c-4831-9bd3-902494283495 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:44.745107 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39162286-7d8c-4831-9bd3-902494283495 admin admin] PATH_INFO: `/v3/projects/76a004cabf7441f985ea431720a1666d/users/d11d11e5e6ec4fbabad193ea95111d11/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:44.761579 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39162286-7d8c-4831-9bd3-902494283495 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76a004cabf7441f985ea431720a1666d, user_id=d11d11e5e6ec4fbabad193ea95111d11, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:44.764346 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39162286-7d8c-4831-9bd3-902494283495 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:44.840459 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1010/2002] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:13:44 2022] POST /identity/v3/users => generated 285 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:13:44.846193 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a2260c-9d1d-411f-b80b-a63fbe1bdfe3 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:44.857947 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a2260c-9d1d-411f-b80b-a63fbe1bdfe3 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:44.860795 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 993/2003] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:13:44 2022] PUT /identity/v3/projects/76a004cabf7441f985ea431720a1666d/users/d11d11e5e6ec4fbabad193ea95111d11/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:13:44.862096 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39162286-7d8c-4831-9bd3-902494283495 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:44.929900 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c79b660-f53b-444b-8ff2-a7b050e118cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:44.931396 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c79b660-f53b-444b-8ff2-a7b050e118cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:44.931682 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c79b660-f53b-444b-8ff2-a7b050e118cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:44.931957 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c79b660-f53b-444b-8ff2-a7b050e118cf admin admin] PATH_INFO: `/v3/users/0b262221a873438cb1600d83026b5dbd` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:44.933139 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39162286-7d8c-4831-9bd3-902494283495 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:44.933337 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c79b660-f53b-444b-8ff2-a7b050e118cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b262221a873438cb1600d83026b5dbd)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:44.935411 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c79b660-f53b-444b-8ff2-a7b050e118cf admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:44.938124 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0a39ff-f65d-4036-9266-0885c9016389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:44.938795 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a39ff-f65d-4036-9266-0885c9016389 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:44.939113 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a39ff-f65d-4036-9266-0885c9016389 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:44.939441 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a39ff-f65d-4036-9266-0885c9016389 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:44.940618 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0a39ff-f65d-4036-9266-0885c9016389 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:44.942671 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0a39ff-f65d-4036-9266-0885c9016389 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:44.945579 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:13:44.955934 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 994/2004] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:13:44.957363 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8133a59-3371-4f73-aee3-035ef7c5412b None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:44.957648 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8133a59-3371-4f73-aee3-035ef7c5412b None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:44.957924 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8133a59-3371-4f73-aee3-035ef7c5412b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:44.989855 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-a8133a59-3371-4f73-aee3-035ef7c5412b None None] MFA Rules not processed for user `d11d11e5e6ec4fbabad193ea95111d11`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:13:44.991082 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-a8133a59-3371-4f73-aee3-035ef7c5412b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:45.056021 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 995/2005] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:13:44 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:13:45.057322 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8133a59-3371-4f73-aee3-035ef7c5412b None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:45.067856 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8133a59-3371-4f73-aee3-035ef7c5412b None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:45.072848 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ab30db-8f61-43e3-9754-09c523be8a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:45.073477 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ab30db-8f61-43e3-9754-09c523be8a9e admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:45.073756 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ab30db-8f61-43e3-9754-09c523be8a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:45.074059 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ab30db-8f61-43e3-9754-09c523be8a9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:45.074658 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ab30db-8f61-43e3-9754-09c523be8a9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:45.076287 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ab30db-8f61-43e3-9754-09c523be8a9e admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:45.086487 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 996/2006] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:13:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:13:45.087454 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ab30db-8f61-43e3-9754-09c523be8a9e admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:45.099696 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ab30db-8f61-43e3-9754-09c523be8a9e admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:45.106402 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cffbf8-40dd-4e74-a5eb-ee2079dc2699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:45.107716 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cffbf8-40dd-4e74-a5eb-ee2079dc2699 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:45.108374 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cffbf8-40dd-4e74-a5eb-ee2079dc2699 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:45.108655 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cffbf8-40dd-4e74-a5eb-ee2079dc2699 admin admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:45.109279 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cffbf8-40dd-4e74-a5eb-ee2079dc2699 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:45.111212 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cffbf8-40dd-4e74-a5eb-ee2079dc2699 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:45.141766 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-9c79b660-f53b-444b-8ff2-a7b050e118cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b262221a873438cb1600d83026b5dbd'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:45.222415 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:13:45.224381 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-9c79b660-f53b-444b-8ff2-a7b050e118cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b262221a873438cb1600d83026b5dbd'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:45.225084 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-9c79b660-f53b-444b-8ff2-a7b050e118cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b262221a873438cb1600d83026b5dbd'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:45.225947 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-9c79b660-f53b-444b-8ff2-a7b050e118cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b262221a873438cb1600d83026b5dbd'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:45.236330 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 997/2007] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:13:45 2022] POST /identity/v3/projects => generated 409 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:13:45.237220 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cffbf8-40dd-4e74-a5eb-ee2079dc2699 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:45.239608 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-08cffbf8-40dd-4e74-a5eb-ee2079dc2699 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:45.240961 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-9c79b660-f53b-444b-8ff2-a7b050e118cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b262221a873438cb1600d83026b5dbd'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:45.260805 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1011/2008] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:13:44 2022] DELETE /identity/v3/users/0b262221a873438cb1600d83026b5dbd => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:13:45.261818 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c79b660-f53b-444b-8ff2-a7b050e118cf admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:45.263235 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-9c79b660-f53b-444b-8ff2-a7b050e118cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:45.311201 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cffbf8-40dd-4e74-a5eb-ee2079dc2699 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:45.344454 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c79b660-f53b-444b-8ff2-a7b050e118cf admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:45.363394 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0666b3-5882-42ec-92b3-26d70706f8b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:45.363823 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0666b3-5882-42ec-92b3-26d70706f8b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:45.364090 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0666b3-5882-42ec-92b3-26d70706f8b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:45.364371 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0666b3-5882-42ec-92b3-26d70706f8b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:45.365013 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0666b3-5882-42ec-92b3-26d70706f8b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:45.377088 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0666b3-5882-42ec-92b3-26d70706f8b9 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:45.391499 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3093312f-5adc-4996-abfb-ce80d17fbbec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:45.392902 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3093312f-5adc-4996-abfb-ce80d17fbbec admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:45.392902 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3093312f-5adc-4996-abfb-ce80d17fbbec admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:45.392902 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3093312f-5adc-4996-abfb-ce80d17fbbec admin admin] PATH_INFO: `/v3/projects/ed818736f2b943c58161a8d5609dfe28/users/3c9baff1a79143199c3b0dc5afb66383/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:45.408673 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3093312f-5adc-4996-abfb-ce80d17fbbec admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed818736f2b943c58161a8d5609dfe28, user_id=3c9baff1a79143199c3b0dc5afb66383, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:45.411407 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3093312f-5adc-4996-abfb-ce80d17fbbec admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:45.556113 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 998/2010] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:13:45 2022] PUT /identity/v3/projects/ed818736f2b943c58161a8d5609dfe28/users/3c9baff1a79143199c3b0dc5afb66383/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:13:45.556320 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1012/2010] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:13:45 2022] POST /identity/v3/users => generated 311 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:13:45.560221 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4386a826-2b92-4b52-ab3e-e96050a1da1a None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:45.560511 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4386a826-2b92-4b52-ab3e-e96050a1da1a None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:45.560789 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4386a826-2b92-4b52-ab3e-e96050a1da1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:45.563943 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0666b3-5882-42ec-92b3-26d70706f8b9 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:45.574426 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0666b3-5882-42ec-92b3-26d70706f8b9 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:45.580051 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-4386a826-2b92-4b52-ab3e-e96050a1da1a None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:13:45.581345 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-4386a826-2b92-4b52-ab3e-e96050a1da1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:45.653339 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30edbe2-8b5c-4d7c-947d-a67d40f2b3e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:45.654107 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30edbe2-8b5c-4d7c-947d-a67d40f2b3e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:45.654633 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30edbe2-8b5c-4d7c-947d-a67d40f2b3e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:45.655146 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30edbe2-8b5c-4d7c-947d-a67d40f2b3e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:45.655744 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30edbe2-8b5c-4d7c-947d-a67d40f2b3e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:45.657312 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 999/2011] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:13:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:13:45.659050 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdebf88-f030-4527-a8ca-352c95a4a739 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:45.659050 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdebf88-f030-4527-a8ca-352c95a4a739 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:45.659343 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdebf88-f030-4527-a8ca-352c95a4a739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:45.660919 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30edbe2-8b5c-4d7c-947d-a67d40f2b3e7 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:45.673968 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1013/2012] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:13:45 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:13:45.677894 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30edbe2-8b5c-4d7c-947d-a67d40f2b3e7 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:45.683846 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-b30edbe2-8b5c-4d7c-947d-a67d40f2b3e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:45.694840 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-2fdebf88-f030-4527-a8ca-352c95a4a739 None None] MFA Rules not processed for user `3c9baff1a79143199c3b0dc5afb66383`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:13:45.697283 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-2fdebf88-f030-4527-a8ca-352c95a4a739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:45.760395 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30edbe2-8b5c-4d7c-947d-a67d40f2b3e7 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:45.765650 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8505f839-4e88-466c-ba32-ca591befff67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:45.766465 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8505f839-4e88-466c-ba32-ca591befff67 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:45.767156 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8505f839-4e88-466c-ba32-ca591befff67 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:45.767331 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8505f839-4e88-466c-ba32-ca591befff67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:45.783238 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1000/2013] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:13:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:13:45.787378 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8505f839-4e88-466c-ba32-ca591befff67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:45.789341 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8505f839-4e88-466c-ba32-ca591befff67 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:45.792868 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdebf88-f030-4527-a8ca-352c95a4a739 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:45.805509 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1014/2014] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:13:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:13:45.807548 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8505f839-4e88-466c-ba32-ca591befff67 service neutron] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:45.810650 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-8505f839-4e88-466c-ba32-ca591befff67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:45.832450 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdebf88-f030-4527-a8ca-352c95a4a739 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:45.840844 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6fe8b2-2a05-4668-8bb5-a842c80410fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:45.841534 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6fe8b2-2a05-4668-8bb5-a842c80410fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:45.841840 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6fe8b2-2a05-4668-8bb5-a842c80410fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:45.842107 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6fe8b2-2a05-4668-8bb5-a842c80410fd admin admin] PATH_INFO: `/v3/projects/f84b4b0ea2fc474c909daef8f8badd45/users/c608540c7d494713bf9574f18dcfdff1/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:45.856143 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:13:45.860963 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6fe8b2-2a05-4668-8bb5-a842c80410fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=f84b4b0ea2fc474c909daef8f8badd45, user_id=c608540c7d494713bf9574f18dcfdff1, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:45.863803 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6fe8b2-2a05-4668-8bb5-a842c80410fd admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:45.907523 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8505f839-4e88-466c-ba32-ca591befff67 service neutron] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:45.911793 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcae262b-3a4a-4c11-b1be-c11bfd8c8d48 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bee1ba6bb324de4915904f83349b3c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:45.912467 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae262b-3a4a-4c11-b1be-c11bfd8c8d48 service glance] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:45.912755 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae262b-3a4a-4c11-b1be-c11bfd8c8d48 service glance] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:45.913029 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae262b-3a4a-4c11-b1be-c11bfd8c8d48 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:45.925064 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcae262b-3a4a-4c11-b1be-c11bfd8c8d48 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:45.926552 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcae262b-3a4a-4c11-b1be-c11bfd8c8d48 service glance] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:45.929061 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1001/2015] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:13:45 2022] PUT /identity/v3/projects/f84b4b0ea2fc474c909daef8f8badd45/users/c608540c7d494713bf9574f18dcfdff1/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:13:45.933410 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6fe8b2-2a05-4668-8bb5-a842c80410fd admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:45.951948 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1015/2016] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:13:45 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:13:46.002975 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6fe8b2-2a05-4668-8bb5-a842c80410fd admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:46.012637 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d549656-526e-427c-9e05-62b2432b9b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:46.013398 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d549656-526e-427c-9e05-62b2432b9b67 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:46.013991 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d549656-526e-427c-9e05-62b2432b9b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:46.014329 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d549656-526e-427c-9e05-62b2432b9b67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:46.015323 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d549656-526e-427c-9e05-62b2432b9b67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:46.017256 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d549656-526e-427c-9e05-62b2432b9b67 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:46.037442 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1002/2017] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:13:45 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 17:13:46.038945 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcae262b-3a4a-4c11-b1be-c11bfd8c8d48 service glance] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:46.051220 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcae262b-3a4a-4c11-b1be-c11bfd8c8d48 service glance] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:46.057892 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9daf7d-74a5-4b9d-a3e4-d95c80b0a8fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:46.062406 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9daf7d-74a5-4b9d-a3e4-d95c80b0a8fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:46.062406 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9daf7d-74a5-4b9d-a3e4-d95c80b0a8fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:46.062406 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9daf7d-74a5-4b9d-a3e4-d95c80b0a8fa admin admin] PATH_INFO: `/v3/projects/f84b4b0ea2fc474c909daef8f8badd45/users/c608540c7d494713bf9574f18dcfdff1/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:46.064498 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9daf7d-74a5-4b9d-a3e4-d95c80b0a8fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=f84b4b0ea2fc474c909daef8f8badd45, user_id=c608540c7d494713bf9574f18dcfdff1, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:46.068223 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9daf7d-74a5-4b9d-a3e4-d95c80b0a8fa admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:46.084898 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1016/2018] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:13:46 2022] PUT /identity/v3/projects/f84b4b0ea2fc474c909daef8f8badd45/users/c608540c7d494713bf9574f18dcfdff1/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:13:46.097514 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26942642-1def-4b40-a7d8-dcf3a7ee0ca5 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:46.097867 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26942642-1def-4b40-a7d8-dcf3a7ee0ca5 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:46.098235 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26942642-1def-4b40-a7d8-dcf3a7ee0ca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:46.131745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-26942642-1def-4b40-a7d8-dcf3a7ee0ca5 None None] MFA Rules not processed for user `c608540c7d494713bf9574f18dcfdff1`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:13:46.132847 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-26942642-1def-4b40-a7d8-dcf3a7ee0ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:46.222459 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1003/2019] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:13:46 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:13:46.229874 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:13:46.244861 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9daf7d-74a5-4b9d-a3e4-d95c80b0a8fa admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:46.247207 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-9d9daf7d-74a5-4b9d-a3e4-d95c80b0a8fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:46.318459 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9daf7d-74a5-4b9d-a3e4-d95c80b0a8fa admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:46.323433 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021ad51c-a89d-4d78-831c-3cc8fd9456fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:46.324391 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ad51c-a89d-4d78-831c-3cc8fd9456fd service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:46.324391 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ad51c-a89d-4d78-831c-3cc8fd9456fd service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:46.324664 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ad51c-a89d-4d78-831c-3cc8fd9456fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:46.334553 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021ad51c-a89d-4d78-831c-3cc8fd9456fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:46.337811 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021ad51c-a89d-4d78-831c-3cc8fd9456fd service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:46.348663 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1017/2020] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:13:46 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:13:46.545555 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:13:46.815733 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26942642-1def-4b40-a7d8-dcf3a7ee0ca5 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:46.853589 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26942642-1def-4b40-a7d8-dcf3a7ee0ca5 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:46.856120 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021ad51c-a89d-4d78-831c-3cc8fd9456fd service cinder] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:46.857812 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-021ad51c-a89d-4d78-831c-3cc8fd9456fd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:46.862512 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0663f726-4c5e-4e34-8a7b-40a487a17d8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:46.863188 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0663f726-4c5e-4e34-8a7b-40a487a17d8d service cinder] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:46.863470 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0663f726-4c5e-4e34-8a7b-40a487a17d8d service cinder] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:46.863740 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0663f726-4c5e-4e34-8a7b-40a487a17d8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:46.877237 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-0663f726-4c5e-4e34-8a7b-40a487a17d8d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:46.956976 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021ad51c-a89d-4d78-831c-3cc8fd9456fd service cinder] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:46.962142 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50dc7b4-c74a-4f04-b455-87b12ac3bc17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:46.963550 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50dc7b4-c74a-4f04-b455-87b12ac3bc17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:46.968117 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50dc7b4-c74a-4f04-b455-87b12ac3bc17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:46.968436 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50dc7b4-c74a-4f04-b455-87b12ac3bc17 admin admin] PATH_INFO: `/v3/projects/5af8b5d29b074a9daacecf15e7c17fa0` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:46.969796 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50dc7b4-c74a-4f04-b455-87b12ac3bc17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5af8b5d29b074a9daacecf15e7c17fa0)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:46.977801 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0663f726-4c5e-4e34-8a7b-40a487a17d8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:46.977801 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0663f726-4c5e-4e34-8a7b-40a487a17d8d service cinder] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:46.982229 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50dc7b4-c74a-4f04-b455-87b12ac3bc17 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:46.990051 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1004/2021] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:13:46 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:13:47.030799 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:13:47.152015 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-c50dc7b4-c74a-4f04-b455-87b12ac3bc17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5af8b5d29b074a9daacecf15e7c17fa0', 'request_id': 'req-c50dc7b4-c74a-4f04-b455-87b12ac3bc17'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:47.172528 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-c50dc7b4-c74a-4f04-b455-87b12ac3bc17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5af8b5d29b074a9daacecf15e7c17fa0', 'request_id': 'req-c50dc7b4-c74a-4f04-b455-87b12ac3bc17'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:47.174147 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:13:47.267715 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-c50dc7b4-c74a-4f04-b455-87b12ac3bc17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5af8b5d29b074a9daacecf15e7c17fa0', 'request_id': 'req-c50dc7b4-c74a-4f04-b455-87b12ac3bc17'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:47.301995 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-c50dc7b4-c74a-4f04-b455-87b12ac3bc17 admin admin] The token cache is being invalidate because project 5af8b5d29b074a9daacecf15e7c17fa0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:13:47.305830 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-c50dc7b4-c74a-4f04-b455-87b12ac3bc17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:47.318615 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-c50dc7b4-c74a-4f04-b455-87b12ac3bc17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:47.322501 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1018/2022] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:13:46 2022] DELETE /identity/v3/projects/5af8b5d29b074a9daacecf15e7c17fa0 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:13:47.326763 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0663f726-4c5e-4e34-8a7b-40a487a17d8d service cinder] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:47.343640 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0663f726-4c5e-4e34-8a7b-40a487a17d8d service cinder] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:47.422721 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a6e08-f324-4feb-b157-de3bb8d66c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:47.423478 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a6e08-f324-4feb-b157-de3bb8d66c6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:47.423829 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a6e08-f324-4feb-b157-de3bb8d66c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:47.424187 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a6e08-f324-4feb-b157-de3bb8d66c6d admin admin] PATH_INFO: `/v3/users/52386e8c2c4e46a9823f419d5e19ed4c` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:47.425392 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1a6e08-f324-4feb-b157-de3bb8d66c6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=52386e8c2c4e46a9823f419d5e19ed4c)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:47.427713 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1a6e08-f324-4feb-b157-de3bb8d66c6d admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:47.855240 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-1f1a6e08-f324-4feb-b157-de3bb8d66c6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52386e8c2c4e46a9823f419d5e19ed4c'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:47.868941 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-1f1a6e08-f324-4feb-b157-de3bb8d66c6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52386e8c2c4e46a9823f419d5e19ed4c'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:47.869733 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-1f1a6e08-f324-4feb-b157-de3bb8d66c6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52386e8c2c4e46a9823f419d5e19ed4c'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:47.949532 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-1f1a6e08-f324-4feb-b157-de3bb8d66c6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52386e8c2c4e46a9823f419d5e19ed4c'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:47.950267 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-1f1a6e08-f324-4feb-b157-de3bb8d66c6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52386e8c2c4e46a9823f419d5e19ed4c'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:47.966234 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1005/2023] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:13:47 2022] DELETE /identity/v3/users/52386e8c2c4e46a9823f419d5e19ed4c => generated 0 bytes in 640 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:13:48.242148 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:13:48.354472 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:13:48.784185 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50dc7b4-c74a-4f04-b455-87b12ac3bc17 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:48.788789 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-c50dc7b4-c74a-4f04-b455-87b12ac3bc17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:48.799536 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50dc7b4-c74a-4f04-b455-87b12ac3bc17 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:48.869762 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b20be10-2a5e-4bce-a582-a4bd107ccb1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:48.870409 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b20be10-2a5e-4bce-a582-a4bd107ccb1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:48.870690 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b20be10-2a5e-4bce-a582-a4bd107ccb1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:48.871050 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b20be10-2a5e-4bce-a582-a4bd107ccb1f admin admin] PATH_INFO: `/v3/projects/8ae4c5ca8064460c9859957e1b05d58b` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:48.872179 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b20be10-2a5e-4bce-a582-a4bd107ccb1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ae4c5ca8064460c9859957e1b05d58b)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:48.874118 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b20be10-2a5e-4bce-a582-a4bd107ccb1f admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:48.977557 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:13:48.980472 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-5b20be10-2a5e-4bce-a582-a4bd107ccb1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ae4c5ca8064460c9859957e1b05d58b', 'request_id': 'req-5b20be10-2a5e-4bce-a582-a4bd107ccb1f'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:48.987369 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-5b20be10-2a5e-4bce-a582-a4bd107ccb1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ae4c5ca8064460c9859957e1b05d58b', 'request_id': 'req-5b20be10-2a5e-4bce-a582-a4bd107ccb1f'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:49.044392 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-5b20be10-2a5e-4bce-a582-a4bd107ccb1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ae4c5ca8064460c9859957e1b05d58b', 'request_id': 'req-5b20be10-2a5e-4bce-a582-a4bd107ccb1f'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:49.057720 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-5b20be10-2a5e-4bce-a582-a4bd107ccb1f admin admin] The token cache is being invalidate because project 8ae4c5ca8064460c9859957e1b05d58b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:13:49.058051 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-5b20be10-2a5e-4bce-a582-a4bd107ccb1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:49.058699 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-5b20be10-2a5e-4bce-a582-a4bd107ccb1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:13:49.065764 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1019/2024] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:13:48 2022] DELETE /identity/v3/projects/8ae4c5ca8064460c9859957e1b05d58b => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:13:49.068702 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:13:49.071790 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9edb0d6-fc40-480e-84c1-2ec51c3f64ce None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:49.075308 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9edb0d6-fc40-480e-84c1-2ec51c3f64ce None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:49.075308 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9edb0d6-fc40-480e-84c1-2ec51c3f64ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:49.093826 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-e9edb0d6-fc40-480e-84c1-2ec51c3f64ce None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:13:49.095100 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-e9edb0d6-fc40-480e-84c1-2ec51c3f64ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:49.203631 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c21d0a2-2c0a-461c-a4c0-4eb02f5a06d9 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:49.203881 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1006/2025] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:13:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:13:49.204325 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c21d0a2-2c0a-461c-a4c0-4eb02f5a06d9 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:49.204763 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c21d0a2-2c0a-461c-a4c0-4eb02f5a06d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:49.232503 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-8c21d0a2-2c0a-461c-a4c0-4eb02f5a06d9 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:13:49.233387 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-8c21d0a2-2c0a-461c-a4c0-4eb02f5a06d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:49.247164 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1020/2026] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:13:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:13:49.247683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9edb0d6-fc40-480e-84c1-2ec51c3f64ce None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:49.260956 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9edb0d6-fc40-480e-84c1-2ec51c3f64ce None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:49.267310 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14687ea9-d07a-4fed-a670-ccaddb4c8651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:49.267909 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14687ea9-d07a-4fed-a670-ccaddb4c8651 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:49.268190 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14687ea9-d07a-4fed-a670-ccaddb4c8651 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:49.268461 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14687ea9-d07a-4fed-a670-ccaddb4c8651 admin admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:49.269610 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14687ea9-d07a-4fed-a670-ccaddb4c8651 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:13:49.269610 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14687ea9-d07a-4fed-a670-ccaddb4c8651 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:49.271073 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14687ea9-d07a-4fed-a670-ccaddb4c8651 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:49.290807 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1007/2027] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:13:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 17:13:49.291099 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c21d0a2-2c0a-461c-a4c0-4eb02f5a06d9 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:49.302792 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c21d0a2-2c0a-461c-a4c0-4eb02f5a06d9 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:49.309731 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee4bfe3-d9b7-4c94-8047-1c0573fea1c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:49.315078 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee4bfe3-d9b7-4c94-8047-1c0573fea1c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:49.315078 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee4bfe3-d9b7-4c94-8047-1c0573fea1c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:49.315078 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee4bfe3-d9b7-4c94-8047-1c0573fea1c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:49.315078 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee4bfe3-d9b7-4c94-8047-1c0573fea1c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:49.315078 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee4bfe3-d9b7-4c94-8047-1c0573fea1c4 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:49.438440 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1021/2028] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:13:49 2022] POST /identity/v3/projects => generated 397 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 17:13:49.442351 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14687ea9-d07a-4fed-a670-ccaddb4c8651 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:49.453837 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14687ea9-d07a-4fed-a670-ccaddb4c8651 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:49.494140 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:13:49.521320 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20349b3f-0562-4ab2-ad83-d5a70a4947de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:49.522209 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20349b3f-0562-4ab2-ad83-d5a70a4947de admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:49.522494 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20349b3f-0562-4ab2-ad83-d5a70a4947de admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:49.522765 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20349b3f-0562-4ab2-ad83-d5a70a4947de admin admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:49.523670 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20349b3f-0562-4ab2-ad83-d5a70a4947de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:49.525627 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20349b3f-0562-4ab2-ad83-d5a70a4947de admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:49.641916 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1008/2029] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:13:49 2022] POST /identity/v3/users => generated 299 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:13:49.646275 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee4bfe3-d9b7-4c94-8047-1c0573fea1c4 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:49.657523 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee4bfe3-d9b7-4c94-8047-1c0573fea1c4 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:49.662771 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816c5cd7-0a83-4229-a187-536c5a3e2902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:49.664559 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816c5cd7-0a83-4229-a187-536c5a3e2902 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:49.665183 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816c5cd7-0a83-4229-a187-536c5a3e2902 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:49.665867 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816c5cd7-0a83-4229-a187-536c5a3e2902 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:49.666711 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816c5cd7-0a83-4229-a187-536c5a3e2902 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:49.668718 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816c5cd7-0a83-4229-a187-536c5a3e2902 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:49.687137 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1022/2030] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:13:49 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:13:49.688837 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20349b3f-0562-4ab2-ad83-d5a70a4947de admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:49.699235 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20349b3f-0562-4ab2-ad83-d5a70a4947de admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:49.717947 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db853f-edee-42f7-888f-a805cb54d8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:49.718363 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db853f-edee-42f7-888f-a805cb54d8e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:49.718790 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db853f-edee-42f7-888f-a805cb54d8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:49.719162 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db853f-edee-42f7-888f-a805cb54d8e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:49.720223 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6db853f-edee-42f7-888f-a805cb54d8e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:49.721977 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6db853f-edee-42f7-888f-a805cb54d8e0 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:49.758188 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1009/2031] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:13:49 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:13:49.762480 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816c5cd7-0a83-4229-a187-536c5a3e2902 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:49.777506 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816c5cd7-0a83-4229-a187-536c5a3e2902 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:49.788102 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d00f82-aa3b-4c11-b7fc-2f5d61213e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:49.792051 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d00f82-aa3b-4c11-b7fc-2f5d61213e86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:49.792637 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d00f82-aa3b-4c11-b7fc-2f5d61213e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:49.792917 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d00f82-aa3b-4c11-b7fc-2f5d61213e86 admin admin] PATH_INFO: `/v3/projects/7ad7d2c8c98d44db87012858a58fda95/users/fbd3e9908cac4f3bbfe6b7c3b13e850f/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:49.820607 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d00f82-aa3b-4c11-b7fc-2f5d61213e86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ad7d2c8c98d44db87012858a58fda95, user_id=fbd3e9908cac4f3bbfe6b7c3b13e850f, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:49.824192 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d00f82-aa3b-4c11-b7fc-2f5d61213e86 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:49.933437 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1023/2032] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:13:49 2022] PUT /identity/v3/projects/7ad7d2c8c98d44db87012858a58fda95/users/fbd3e9908cac4f3bbfe6b7c3b13e850f/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:13:49.941539 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b0b234-6a79-4d60-a6a2-49a25fc5e755 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:49.944358 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b0b234-6a79-4d60-a6a2-49a25fc5e755 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:49.944358 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b0b234-6a79-4d60-a6a2-49a25fc5e755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:49.970373 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-55b0b234-6a79-4d60-a6a2-49a25fc5e755 None None] MFA Rules not processed for user `fbd3e9908cac4f3bbfe6b7c3b13e850f`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:13:49.972965 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-55b0b234-6a79-4d60-a6a2-49a25fc5e755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:50.029207 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:13:50.044391 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1010/2033] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:13:49 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:13:50.049351 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d00f82-aa3b-4c11-b7fc-2f5d61213e86 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:50.120452 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d00f82-aa3b-4c11-b7fc-2f5d61213e86 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:50.125835 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbc4fd4-fc22-4464-b683-797d0e1c2a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:50.126653 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc4fd4-fc22-4464-b683-797d0e1c2a12 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:50.126961 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc4fd4-fc22-4464-b683-797d0e1c2a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:50.127237 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc4fd4-fc22-4464-b683-797d0e1c2a12 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:50.127877 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbc4fd4-fc22-4464-b683-797d0e1c2a12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:50.129912 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbc4fd4-fc22-4464-b683-797d0e1c2a12 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:50.243122 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:13:50.252733 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1024/2034] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:13:50 2022] POST /identity/v3/projects => generated 401 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 17:13:50.253850 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b0b234-6a79-4d60-a6a2-49a25fc5e755 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:50.279288 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b0b234-6a79-4d60-a6a2-49a25fc5e755 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:50.339496 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab024e3-7a9f-464a-b094-99132d3b92db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:50.340036 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab024e3-7a9f-464a-b094-99132d3b92db admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:50.340315 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab024e3-7a9f-464a-b094-99132d3b92db admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:50.340587 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab024e3-7a9f-464a-b094-99132d3b92db admin admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:50.341250 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab024e3-7a9f-464a-b094-99132d3b92db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:50.343227 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab024e3-7a9f-464a-b094-99132d3b92db admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:50.511012 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1011/2035] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:13:50 2022] POST /identity/v3/users => generated 307 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 17:13:50.514462 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbc4fd4-fc22-4464-b683-797d0e1c2a12 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:50.524931 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbc4fd4-fc22-4464-b683-797d0e1c2a12 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:50.529614 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a85d85e-0d74-4824-89a8-b4009e6a272c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:50.530213 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a85d85e-0d74-4824-89a8-b4009e6a272c admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:50.530495 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a85d85e-0d74-4824-89a8-b4009e6a272c admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:50.530853 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a85d85e-0d74-4824-89a8-b4009e6a272c admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:50.531494 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a85d85e-0d74-4824-89a8-b4009e6a272c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:50.533463 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a85d85e-0d74-4824-89a8-b4009e6a272c admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:50.544735 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1025/2036] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:13:50.548963 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab024e3-7a9f-464a-b094-99132d3b92db admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:50.559576 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab024e3-7a9f-464a-b094-99132d3b92db admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:50.564132 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d75a2d-0d03-415a-9554-a713f7062b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:50.564784 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d75a2d-0d03-415a-9554-a713f7062b2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:50.565067 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d75a2d-0d03-415a-9554-a713f7062b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:50.565340 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d75a2d-0d03-415a-9554-a713f7062b2b admin admin] PATH_INFO: `/v3/projects/9c1ce2ef86224787a1a91bf3e8a3b778/users/6e355b192d2a4301bf764fa2f672ad25/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:50.580094 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d75a2d-0d03-415a-9554-a713f7062b2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c1ce2ef86224787a1a91bf3e8a3b778, user_id=6e355b192d2a4301bf764fa2f672ad25, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:50.587467 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d75a2d-0d03-415a-9554-a713f7062b2b admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:50.610352 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:13:50.692211 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1012/2037] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:13:50 2022] PUT /identity/v3/projects/9c1ce2ef86224787a1a91bf3e8a3b778/users/6e355b192d2a4301bf764fa2f672ad25/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:13:50.697032 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a85d85e-0d74-4824-89a8-b4009e6a272c admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:50.706670 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a85d85e-0d74-4824-89a8-b4009e6a272c admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:50.767905 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4f604e-a522-4123-9ee1-45863634e004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:50.768588 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4f604e-a522-4123-9ee1-45863634e004 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:50.768973 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4f604e-a522-4123-9ee1-45863634e004 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:50.769356 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4f604e-a522-4123-9ee1-45863634e004 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:50.770101 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4f604e-a522-4123-9ee1-45863634e004 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:50.771773 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4f604e-a522-4123-9ee1-45863634e004 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:50.782809 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1026/2038] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:13:50.786807 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d75a2d-0d03-415a-9554-a713f7062b2b admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:50.797875 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d75a2d-0d03-415a-9554-a713f7062b2b admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:50.805953 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56b2fdd-10e1-4bbc-b695-44578f8ea1ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:50.806620 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b2fdd-10e1-4bbc-b695-44578f8ea1ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:50.806926 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b2fdd-10e1-4bbc-b695-44578f8ea1ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:50.807217 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b2fdd-10e1-4bbc-b695-44578f8ea1ad admin admin] PATH_INFO: `/v3/projects/9c1ce2ef86224787a1a91bf3e8a3b778/users/6e355b192d2a4301bf764fa2f672ad25/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:50.809289 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56b2fdd-10e1-4bbc-b695-44578f8ea1ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c1ce2ef86224787a1a91bf3e8a3b778, user_id=6e355b192d2a4301bf764fa2f672ad25, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:50.811955 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56b2fdd-10e1-4bbc-b695-44578f8ea1ad admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:50.822152 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1013/2039] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:13:50 2022] PUT /identity/v3/projects/9c1ce2ef86224787a1a91bf3e8a3b778/users/6e355b192d2a4301bf764fa2f672ad25/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:13:50.832304 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a331967-2db3-4718-8833-ffef22d299e1 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:50.840759 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a331967-2db3-4718-8833-ffef22d299e1 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:50.840759 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a331967-2db3-4718-8833-ffef22d299e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:50.860933 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-2a331967-2db3-4718-8833-ffef22d299e1 None None] MFA Rules not processed for user `6e355b192d2a4301bf764fa2f672ad25`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:13:50.862067 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-2a331967-2db3-4718-8833-ffef22d299e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:50.930412 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1027/2040] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:13:50 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:13:50.942106 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56b2fdd-10e1-4bbc-b695-44578f8ea1ad admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:50.943893 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-b56b2fdd-10e1-4bbc-b695-44578f8ea1ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:51.002108 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56b2fdd-10e1-4bbc-b695-44578f8ea1ad admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:51.005879 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb8ff53-72b8-488b-82b5-44d09a43fc56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:51.006499 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb8ff53-72b8-488b-82b5-44d09a43fc56 service cinder] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:51.006778 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb8ff53-72b8-488b-82b5-44d09a43fc56 service cinder] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:51.007093 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb8ff53-72b8-488b-82b5-44d09a43fc56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:51.017279 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb8ff53-72b8-488b-82b5-44d09a43fc56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:51.018716 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb8ff53-72b8-488b-82b5-44d09a43fc56 service cinder] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:51.029651 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1014/2041] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:13:50 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:13:51.245023 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a331967-2db3-4718-8833-ffef22d299e1 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:13:51.246489 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-2a331967-2db3-4718-8833-ffef22d299e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:13:51.259138 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a331967-2db3-4718-8833-ffef22d299e1 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:13:51.262758 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7377ef8-d04a-46d0-8792-1122483eda72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:13:51.263413 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7377ef8-d04a-46d0-8792-1122483eda72 service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:13:51.264349 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7377ef8-d04a-46d0-8792-1122483eda72 service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:13:51.264632 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7377ef8-d04a-46d0-8792-1122483eda72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:13:51.328911 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7377ef8-d04a-46d0-8792-1122483eda72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:13:51.330354 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7377ef8-d04a-46d0-8792-1122483eda72 service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:13:51.339656 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1028/2042] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:13:51 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:14:01.025708 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=66,keystone:INSERT=5 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:01.335541 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=96 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:02.413232 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb8ff53-72b8-488b-82b5-44d09a43fc56 service cinder] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:02.414619 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-ceb8ff53-72b8-488b-82b5-44d09a43fc56 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:02.480861 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb8ff53-72b8-488b-82b5-44d09a43fc56 service cinder] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:02.487335 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c8f587-c535-4e2d-b553-3a7e4b9938e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:02.487973 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c8f587-c535-4e2d-b553-3a7e4b9938e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:02.488257 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c8f587-c535-4e2d-b553-3a7e4b9938e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:02.488532 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c8f587-c535-4e2d-b553-3a7e4b9938e5 admin admin] PATH_INFO: `/v3/users/94d03f8b4756475ca1502e100203e947` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:02.489659 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c8f587-c535-4e2d-b553-3a7e4b9938e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94d03f8b4756475ca1502e100203e947)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:02.491685 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c8f587-c535-4e2d-b553-3a7e4b9938e5 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:02.683571 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-47c8f587-c535-4e2d-b553-3a7e4b9938e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94d03f8b4756475ca1502e100203e947'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:02.695260 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-47c8f587-c535-4e2d-b553-3a7e4b9938e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94d03f8b4756475ca1502e100203e947'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:02.695971 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-47c8f587-c535-4e2d-b553-3a7e4b9938e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94d03f8b4756475ca1502e100203e947'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:02.746296 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-47c8f587-c535-4e2d-b553-3a7e4b9938e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94d03f8b4756475ca1502e100203e947'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:02.746928 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-47c8f587-c535-4e2d-b553-3a7e4b9938e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94d03f8b4756475ca1502e100203e947'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:02.761832 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1015/2043] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:14:02 2022] DELETE /identity/v3/users/94d03f8b4756475ca1502e100203e947 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:14:02.767734 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754cdc4a-5190-4a14-88e5-efd8d0a11441 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:02.768171 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754cdc4a-5190-4a14-88e5-efd8d0a11441 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:02.768438 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754cdc4a-5190-4a14-88e5-efd8d0a11441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:02.792456 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-754cdc4a-5190-4a14-88e5-efd8d0a11441 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:14:02.793665 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-754cdc4a-5190-4a14-88e5-efd8d0a11441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:02.859989 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1029/2044] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:14:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:14:02.870480 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c8f587-c535-4e2d-b553-3a7e4b9938e5 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:02.872105 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-47c8f587-c535-4e2d-b553-3a7e4b9938e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:02.930378 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c8f587-c535-4e2d-b553-3a7e4b9938e5 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:02.935615 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfe388d-9537-41fb-b86d-4c1b07023fe3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:02.936223 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfe388d-9537-41fb-b86d-4c1b07023fe3 service neutron] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:02.936505 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfe388d-9537-41fb-b86d-4c1b07023fe3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:02.936788 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfe388d-9537-41fb-b86d-4c1b07023fe3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:02.946264 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfe388d-9537-41fb-b86d-4c1b07023fe3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:02.947977 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfe388d-9537-41fb-b86d-4c1b07023fe3 service neutron] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:02.957696 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1016/2045] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:14:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:14:03.360961 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754cdc4a-5190-4a14-88e5-efd8d0a11441 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:03.362480 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-754cdc4a-5190-4a14-88e5-efd8d0a11441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:03.376552 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754cdc4a-5190-4a14-88e5-efd8d0a11441 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:03.381459 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a46fc9-2ad9-438d-b7d9-7f9eac3b7930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:03.382527 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a46fc9-2ad9-438d-b7d9-7f9eac3b7930 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:03.382809 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a46fc9-2ad9-438d-b7d9-7f9eac3b7930 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:03.383944 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a46fc9-2ad9-438d-b7d9-7f9eac3b7930 admin admin] PATH_INFO: `/v3/projects/4529421fbf2445259eb7dde6b2df831a` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:03.385866 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a46fc9-2ad9-438d-b7d9-7f9eac3b7930 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4529421fbf2445259eb7dde6b2df831a)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:03.398057 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a46fc9-2ad9-438d-b7d9-7f9eac3b7930 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:03.503024 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-97a46fc9-2ad9-438d-b7d9-7f9eac3b7930 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4529421fbf2445259eb7dde6b2df831a', 'request_id': 'req-97a46fc9-2ad9-438d-b7d9-7f9eac3b7930'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:03.509953 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-97a46fc9-2ad9-438d-b7d9-7f9eac3b7930 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4529421fbf2445259eb7dde6b2df831a', 'request_id': 'req-97a46fc9-2ad9-438d-b7d9-7f9eac3b7930'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:03.577124 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-97a46fc9-2ad9-438d-b7d9-7f9eac3b7930 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4529421fbf2445259eb7dde6b2df831a', 'request_id': 'req-97a46fc9-2ad9-438d-b7d9-7f9eac3b7930'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:03.591311 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-97a46fc9-2ad9-438d-b7d9-7f9eac3b7930 admin admin] The token cache is being invalidate because project 4529421fbf2445259eb7dde6b2df831a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:14:03.591608 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-97a46fc9-2ad9-438d-b7d9-7f9eac3b7930 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:03.592264 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-97a46fc9-2ad9-438d-b7d9-7f9eac3b7930 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:03.593797 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1030/2046] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:14:03 2022] DELETE /identity/v3/projects/4529421fbf2445259eb7dde6b2df831a => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:14:03.598013 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfe388d-9537-41fb-b86d-4c1b07023fe3 service neutron] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:03.607920 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfe388d-9537-41fb-b86d-4c1b07023fe3 service neutron] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:03.626096 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:03.664188 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b79fd4f-86ea-4d18-bcce-82653b621328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:03.664853 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79fd4f-86ea-4d18-bcce-82653b621328 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:03.665146 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79fd4f-86ea-4d18-bcce-82653b621328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:03.665421 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79fd4f-86ea-4d18-bcce-82653b621328 admin admin] PATH_INFO: `/v3/users/97144aca51ae4154b43febf8611e2b23` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:03.666767 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b79fd4f-86ea-4d18-bcce-82653b621328 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97144aca51ae4154b43febf8611e2b23)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:03.668730 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b79fd4f-86ea-4d18-bcce-82653b621328 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:03.864238 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-7b79fd4f-86ea-4d18-bcce-82653b621328 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97144aca51ae4154b43febf8611e2b23'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:03.876743 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-7b79fd4f-86ea-4d18-bcce-82653b621328 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97144aca51ae4154b43febf8611e2b23'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:03.877433 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-7b79fd4f-86ea-4d18-bcce-82653b621328 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97144aca51ae4154b43febf8611e2b23'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:03.940237 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-7b79fd4f-86ea-4d18-bcce-82653b621328 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97144aca51ae4154b43febf8611e2b23'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:03.940900 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-7b79fd4f-86ea-4d18-bcce-82653b621328 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97144aca51ae4154b43febf8611e2b23'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:03.954734 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1017/2047] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:14:03 2022] DELETE /identity/v3/users/97144aca51ae4154b43febf8611e2b23 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:14:04.441578 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a46fc9-2ad9-438d-b7d9-7f9eac3b7930 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:04.443089 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-97a46fc9-2ad9-438d-b7d9-7f9eac3b7930 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:04.456945 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a46fc9-2ad9-438d-b7d9-7f9eac3b7930 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:04.517754 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e736203-a67d-461e-9a96-419e6056273c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:04.518318 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e736203-a67d-461e-9a96-419e6056273c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:04.518596 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e736203-a67d-461e-9a96-419e6056273c admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:04.518868 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e736203-a67d-461e-9a96-419e6056273c admin admin] PATH_INFO: `/v3/projects/6e4c0c3313ea490e948b7353d0cd3ce6` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:04.520017 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e736203-a67d-461e-9a96-419e6056273c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e4c0c3313ea490e948b7353d0cd3ce6)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:04.522070 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e736203-a67d-461e-9a96-419e6056273c admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:04.612939 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:04.629442 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-1e736203-a67d-461e-9a96-419e6056273c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e4c0c3313ea490e948b7353d0cd3ce6', 'request_id': 'req-1e736203-a67d-461e-9a96-419e6056273c'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:04.637374 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-1e736203-a67d-461e-9a96-419e6056273c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e4c0c3313ea490e948b7353d0cd3ce6', 'request_id': 'req-1e736203-a67d-461e-9a96-419e6056273c'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:04.696833 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-1e736203-a67d-461e-9a96-419e6056273c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e4c0c3313ea490e948b7353d0cd3ce6', 'request_id': 'req-1e736203-a67d-461e-9a96-419e6056273c'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:04.707823 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-1e736203-a67d-461e-9a96-419e6056273c admin admin] The token cache is being invalidate because project 6e4c0c3313ea490e948b7353d0cd3ce6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:14:04.708090 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-1e736203-a67d-461e-9a96-419e6056273c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:04.708709 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-1e736203-a67d-461e-9a96-419e6056273c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:04.710159 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1031/2048] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:14:04 2022] DELETE /identity/v3/projects/6e4c0c3313ea490e948b7353d0cd3ce6 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:14:04.714681 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b79fd4f-86ea-4d18-bcce-82653b621328 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:04.716206 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-7b79fd4f-86ea-4d18-bcce-82653b621328 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:04.729607 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b79fd4f-86ea-4d18-bcce-82653b621328 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:04.787086 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebd9c62-80ad-4127-baaa-84e4497f328a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:04.787727 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebd9c62-80ad-4127-baaa-84e4497f328a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:04.788057 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebd9c62-80ad-4127-baaa-84e4497f328a admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:04.788449 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebd9c62-80ad-4127-baaa-84e4497f328a admin admin] PATH_INFO: `/v3/users/65ed8341b4ce48418e6bb0374da39373` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:04.789604 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebd9c62-80ad-4127-baaa-84e4497f328a admin admin] RBAC: Authorizing `identity:delete_user(user_id=65ed8341b4ce48418e6bb0374da39373)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:04.792423 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebd9c62-80ad-4127-baaa-84e4497f328a admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:05.010972 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:05.013142 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-6ebd9c62-80ad-4127-baaa-84e4497f328a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65ed8341b4ce48418e6bb0374da39373'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:05.025258 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-6ebd9c62-80ad-4127-baaa-84e4497f328a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65ed8341b4ce48418e6bb0374da39373'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:05.025921 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-6ebd9c62-80ad-4127-baaa-84e4497f328a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65ed8341b4ce48418e6bb0374da39373'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:05.102952 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-6ebd9c62-80ad-4127-baaa-84e4497f328a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65ed8341b4ce48418e6bb0374da39373'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:05.103656 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-6ebd9c62-80ad-4127-baaa-84e4497f328a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65ed8341b4ce48418e6bb0374da39373'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:05.117019 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1018/2049] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:14:04 2022] DELETE /identity/v3/users/65ed8341b4ce48418e6bb0374da39373 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:14:05.506696 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e736203-a67d-461e-9a96-419e6056273c admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:05.509433 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-1e736203-a67d-461e-9a96-419e6056273c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:05.523717 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e736203-a67d-461e-9a96-419e6056273c admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:05.584791 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9e9d04-c04a-431b-8d32-9feac7f14f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:05.585381 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9e9d04-c04a-431b-8d32-9feac7f14f04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:05.585663 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9e9d04-c04a-431b-8d32-9feac7f14f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:05.585936 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9e9d04-c04a-431b-8d32-9feac7f14f04 admin admin] PATH_INFO: `/v3/projects/7982c59fbbe4438aa53b45684e0fa5e3` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:05.587219 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9e9d04-c04a-431b-8d32-9feac7f14f04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7982c59fbbe4438aa53b45684e0fa5e3)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:05.589064 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9e9d04-c04a-431b-8d32-9feac7f14f04 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:05.725607 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-8a9e9d04-c04a-431b-8d32-9feac7f14f04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7982c59fbbe4438aa53b45684e0fa5e3', 'request_id': 'req-8a9e9d04-c04a-431b-8d32-9feac7f14f04'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:05.732341 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-8a9e9d04-c04a-431b-8d32-9feac7f14f04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7982c59fbbe4438aa53b45684e0fa5e3', 'request_id': 'req-8a9e9d04-c04a-431b-8d32-9feac7f14f04'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:05.808937 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-8a9e9d04-c04a-431b-8d32-9feac7f14f04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7982c59fbbe4438aa53b45684e0fa5e3', 'request_id': 'req-8a9e9d04-c04a-431b-8d32-9feac7f14f04'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:05.819815 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-8a9e9d04-c04a-431b-8d32-9feac7f14f04 admin admin] The token cache is being invalidate because project 7982c59fbbe4438aa53b45684e0fa5e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:14:05.820097 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-8a9e9d04-c04a-431b-8d32-9feac7f14f04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:05.820749 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-8a9e9d04-c04a-431b-8d32-9feac7f14f04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:05.822417 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1032/2050] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:14:05 2022] DELETE /identity/v3/projects/7982c59fbbe4438aa53b45684e0fa5e3 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:14:05.827309 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2ee47c-5e58-4096-a269-344635b9c442 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:05.827780 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2ee47c-5e58-4096-a269-344635b9c442 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:05.827884 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2ee47c-5e58-4096-a269-344635b9c442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:05.847403 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-dd2ee47c-5e58-4096-a269-344635b9c442 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:14:05.848533 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-dd2ee47c-5e58-4096-a269-344635b9c442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:05.908051 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1019/2051] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:14:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:14:05.916848 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a11b5-9005-43cb-9d5f-f5c4c4eb2ae3 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:05.917129 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a11b5-9005-43cb-9d5f-f5c4c4eb2ae3 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:05.917402 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a11b5-9005-43cb-9d5f-f5c4c4eb2ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:05.937163 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-3e4a11b5-9005-43cb-9d5f-f5c4c4eb2ae3 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:14:05.938365 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-3e4a11b5-9005-43cb-9d5f-f5c4c4eb2ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:05.946145 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1033/2052] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:14:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:14:05.950296 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2ee47c-5e58-4096-a269-344635b9c442 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:05.960010 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2ee47c-5e58-4096-a269-344635b9c442 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:05.964512 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf0c1cc-1ffa-4759-8120-5c13742a52c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:05.965112 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf0c1cc-1ffa-4759-8120-5c13742a52c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:05.965395 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf0c1cc-1ffa-4759-8120-5c13742a52c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:05.965669 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf0c1cc-1ffa-4759-8120-5c13742a52c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:05.966307 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf0c1cc-1ffa-4759-8120-5c13742a52c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:14:05.966610 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf0c1cc-1ffa-4759-8120-5c13742a52c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:05.968253 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf0c1cc-1ffa-4759-8120-5c13742a52c7 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:05.980219 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1020/2053] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:14:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 17:14:05.984428 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4a11b5-9005-43cb-9d5f-f5c4c4eb2ae3 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:05.994216 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4a11b5-9005-43cb-9d5f-f5c4c4eb2ae3 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:05.998846 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc242de-2efe-49ee-8829-44487cd28a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:05.999472 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc242de-2efe-49ee-8829-44487cd28a0a admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:05.999753 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc242de-2efe-49ee-8829-44487cd28a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:06.000026 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc242de-2efe-49ee-8829-44487cd28a0a admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:06.000635 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc242de-2efe-49ee-8829-44487cd28a0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:06.002519 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc242de-2efe-49ee-8829-44487cd28a0a admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:06.112562 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1034/2054] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:14:05 2022] POST /identity/v3/projects => generated 399 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:14:06.116626 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf0c1cc-1ffa-4759-8120-5c13742a52c7 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:06.126620 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf0c1cc-1ffa-4759-8120-5c13742a52c7 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:06.183332 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8852cd6d-7957-41c0-b472-c5d418b73467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:06.183945 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8852cd6d-7957-41c0-b472-c5d418b73467 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:06.184226 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8852cd6d-7957-41c0-b472-c5d418b73467 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:06.184496 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8852cd6d-7957-41c0-b472-c5d418b73467 admin admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:06.185111 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8852cd6d-7957-41c0-b472-c5d418b73467 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:06.187110 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8852cd6d-7957-41c0-b472-c5d418b73467 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:06.283210 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1021/2055] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:14:06 2022] POST /identity/v3/users => generated 300 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 17:14:06.287592 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc242de-2efe-49ee-8829-44487cd28a0a admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:06.299187 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc242de-2efe-49ee-8829-44487cd28a0a admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:06.303982 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cfaf60-85d2-4ae7-865a-dcbaaa42aa7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:06.304591 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cfaf60-85d2-4ae7-865a-dcbaaa42aa7b admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:06.304921 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cfaf60-85d2-4ae7-865a-dcbaaa42aa7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:06.305203 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cfaf60-85d2-4ae7-865a-dcbaaa42aa7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:06.305939 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cfaf60-85d2-4ae7-865a-dcbaaa42aa7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:06.307595 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cfaf60-85d2-4ae7-865a-dcbaaa42aa7b admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:06.317856 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1035/2056] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:14:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 17:14:06.322357 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8852cd6d-7957-41c0-b472-c5d418b73467 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:06.332972 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8852cd6d-7957-41c0-b472-c5d418b73467 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:06.338723 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d535d87d-dae5-4f21-a57d-6a55f60ba080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:06.339277 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535d87d-dae5-4f21-a57d-6a55f60ba080 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:06.339560 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535d87d-dae5-4f21-a57d-6a55f60ba080 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:06.339839 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535d87d-dae5-4f21-a57d-6a55f60ba080 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:06.340481 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d535d87d-dae5-4f21-a57d-6a55f60ba080 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:06.342095 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d535d87d-dae5-4f21-a57d-6a55f60ba080 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:06.354271 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1022/2057] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:14:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:14:06.359724 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cfaf60-85d2-4ae7-865a-dcbaaa42aa7b admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:06.366242 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:06.370141 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cfaf60-85d2-4ae7-865a-dcbaaa42aa7b admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:06.375214 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44650091-b6ba-410a-8719-146ba22311dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:06.384634 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44650091-b6ba-410a-8719-146ba22311dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:06.384634 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44650091-b6ba-410a-8719-146ba22311dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:06.384634 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44650091-b6ba-410a-8719-146ba22311dd admin admin] PATH_INFO: `/v3/projects/3d406a416a48402588b0c57278139ad4/users/f515adcb16b64ca88d930abc1840523c/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:06.393761 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44650091-b6ba-410a-8719-146ba22311dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d406a416a48402588b0c57278139ad4, user_id=f515adcb16b64ca88d930abc1840523c, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:06.397645 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44650091-b6ba-410a-8719-146ba22311dd admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:06.451328 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1036/2058] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:14:06 2022] PUT /identity/v3/projects/3d406a416a48402588b0c57278139ad4/users/f515adcb16b64ca88d930abc1840523c/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:14:06.462415 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55801e5-899a-4841-b719-2903bc0eb596 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:06.463077 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55801e5-899a-4841-b719-2903bc0eb596 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:06.463077 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55801e5-899a-4841-b719-2903bc0eb596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:06.491657 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-a55801e5-899a-4841-b719-2903bc0eb596 None None] MFA Rules not processed for user `f515adcb16b64ca88d930abc1840523c`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:14:06.492891 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-a55801e5-899a-4841-b719-2903bc0eb596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:06.505167 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:06.559180 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1023/2059] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:14:06 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:14:06.571510 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44650091-b6ba-410a-8719-146ba22311dd admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:06.573369 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-44650091-b6ba-410a-8719-146ba22311dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:06.630515 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44650091-b6ba-410a-8719-146ba22311dd admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:06.634730 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb38bfb4-c8e9-43fa-bafe-774949971541 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:06.635368 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb38bfb4-c8e9-43fa-bafe-774949971541 service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:06.635690 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb38bfb4-c8e9-43fa-bafe-774949971541 service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:06.635971 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb38bfb4-c8e9-43fa-bafe-774949971541 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:06.644061 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb38bfb4-c8e9-43fa-bafe-774949971541 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:06.645521 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb38bfb4-c8e9-43fa-bafe-774949971541 service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:06.654787 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1037/2060] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:14:06 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:14:09.244807 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55801e5-899a-4841-b719-2903bc0eb596 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:09.246276 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-a55801e5-899a-4841-b719-2903bc0eb596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:09.312673 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55801e5-899a-4841-b719-2903bc0eb596 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:09.317460 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e0590a-b9c9-40f4-8a7f-dfe59cc869b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:09.318100 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e0590a-b9c9-40f4-8a7f-dfe59cc869b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:09.318382 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e0590a-b9c9-40f4-8a7f-dfe59cc869b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:09.318657 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e0590a-b9c9-40f4-8a7f-dfe59cc869b9 admin admin] PATH_INFO: `/v3/users/fbd3e9908cac4f3bbfe6b7c3b13e850f` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:09.319849 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e0590a-b9c9-40f4-8a7f-dfe59cc869b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbd3e9908cac4f3bbfe6b7c3b13e850f)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:09.322180 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e0590a-b9c9-40f4-8a7f-dfe59cc869b9 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:09.548136 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-51e0590a-b9c9-40f4-8a7f-dfe59cc869b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbd3e9908cac4f3bbfe6b7c3b13e850f'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:09.562680 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-51e0590a-b9c9-40f4-8a7f-dfe59cc869b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbd3e9908cac4f3bbfe6b7c3b13e850f'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:09.563740 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-51e0590a-b9c9-40f4-8a7f-dfe59cc869b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbd3e9908cac4f3bbfe6b7c3b13e850f'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:09.651211 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-51e0590a-b9c9-40f4-8a7f-dfe59cc869b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbd3e9908cac4f3bbfe6b7c3b13e850f'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:09.660413 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-51e0590a-b9c9-40f4-8a7f-dfe59cc869b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbd3e9908cac4f3bbfe6b7c3b13e850f'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:09.665278 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1024/2061] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:14:09 2022] DELETE /identity/v3/users/fbd3e9908cac4f3bbfe6b7c3b13e850f => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:14:09.670180 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aad38ac-2635-48b5-8f89-82928f3e306e None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:09.670716 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aad38ac-2635-48b5-8f89-82928f3e306e None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:09.670716 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aad38ac-2635-48b5-8f89-82928f3e306e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:09.690477 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-9aad38ac-2635-48b5-8f89-82928f3e306e None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:14:09.691729 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-9aad38ac-2635-48b5-8f89-82928f3e306e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:09.770286 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1038/2062] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:14:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:14:09.781419 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e0590a-b9c9-40f4-8a7f-dfe59cc869b9 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:09.783217 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-51e0590a-b9c9-40f4-8a7f-dfe59cc869b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:09.846117 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:09.863179 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e0590a-b9c9-40f4-8a7f-dfe59cc869b9 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:09.866922 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1b8321-85e7-4424-8239-310ae7f27cbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:09.868124 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b8321-85e7-4424-8239-310ae7f27cbf service neutron] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:09.868635 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b8321-85e7-4424-8239-310ae7f27cbf service neutron] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:09.868910 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b8321-85e7-4424-8239-310ae7f27cbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:09.879618 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1b8321-85e7-4424-8239-310ae7f27cbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:09.881162 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1b8321-85e7-4424-8239-310ae7f27cbf service neutron] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:09.892444 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1025/2063] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:14:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:14:10.400508 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aad38ac-2635-48b5-8f89-82928f3e306e None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:10.402050 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-9aad38ac-2635-48b5-8f89-82928f3e306e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:10.420875 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aad38ac-2635-48b5-8f89-82928f3e306e None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:10.426418 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327798b5-c405-45ef-8920-66f95f74bc8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:10.427146 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327798b5-c405-45ef-8920-66f95f74bc8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:10.427610 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327798b5-c405-45ef-8920-66f95f74bc8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:10.427860 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327798b5-c405-45ef-8920-66f95f74bc8c admin admin] PATH_INFO: `/v3/projects/7ad7d2c8c98d44db87012858a58fda95` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:10.429258 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327798b5-c405-45ef-8920-66f95f74bc8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ad7d2c8c98d44db87012858a58fda95)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:10.432675 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327798b5-c405-45ef-8920-66f95f74bc8c admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:10.567366 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-327798b5-c405-45ef-8920-66f95f74bc8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ad7d2c8c98d44db87012858a58fda95', 'request_id': 'req-327798b5-c405-45ef-8920-66f95f74bc8c'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:10.573768 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-327798b5-c405-45ef-8920-66f95f74bc8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ad7d2c8c98d44db87012858a58fda95', 'request_id': 'req-327798b5-c405-45ef-8920-66f95f74bc8c'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:10.639374 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-327798b5-c405-45ef-8920-66f95f74bc8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ad7d2c8c98d44db87012858a58fda95', 'request_id': 'req-327798b5-c405-45ef-8920-66f95f74bc8c'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:10.656597 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-327798b5-c405-45ef-8920-66f95f74bc8c admin admin] The token cache is being invalidate because project 7ad7d2c8c98d44db87012858a58fda95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:14:10.656597 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-327798b5-c405-45ef-8920-66f95f74bc8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:10.656597 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-327798b5-c405-45ef-8920-66f95f74bc8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:10.656597 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1039/2064] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:14:10 2022] DELETE /identity/v3/projects/7ad7d2c8c98d44db87012858a58fda95 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:14:10.659312 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1b8321-85e7-4424-8239-310ae7f27cbf service neutron] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:10.669931 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1b8321-85e7-4424-8239-310ae7f27cbf service neutron] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:10.726284 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6c303f-ac7b-477d-aad4-9019879b7b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:10.727587 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6c303f-ac7b-477d-aad4-9019879b7b85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:10.727935 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6c303f-ac7b-477d-aad4-9019879b7b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:10.728143 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6c303f-ac7b-477d-aad4-9019879b7b85 admin admin] PATH_INFO: `/v3/users/6e355b192d2a4301bf764fa2f672ad25` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:10.729293 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6c303f-ac7b-477d-aad4-9019879b7b85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e355b192d2a4301bf764fa2f672ad25)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:10.731237 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6c303f-ac7b-477d-aad4-9019879b7b85 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:10.934228 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-ba6c303f-ac7b-477d-aad4-9019879b7b85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e355b192d2a4301bf764fa2f672ad25'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:10.946213 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-ba6c303f-ac7b-477d-aad4-9019879b7b85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e355b192d2a4301bf764fa2f672ad25'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:10.953637 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-ba6c303f-ac7b-477d-aad4-9019879b7b85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e355b192d2a4301bf764fa2f672ad25'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:11.028154 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-ba6c303f-ac7b-477d-aad4-9019879b7b85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e355b192d2a4301bf764fa2f672ad25'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:11.028792 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-ba6c303f-ac7b-477d-aad4-9019879b7b85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e355b192d2a4301bf764fa2f672ad25'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:11.044479 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1026/2065] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:14:10 2022] DELETE /identity/v3/users/6e355b192d2a4301bf764fa2f672ad25 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:14:11.518923 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327798b5-c405-45ef-8920-66f95f74bc8c admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:11.526640 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-327798b5-c405-45ef-8920-66f95f74bc8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:11.528088 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6c303f-ac7b-477d-aad4-9019879b7b85 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:11.529840 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-ba6c303f-ac7b-477d-aad4-9019879b7b85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:11.552507 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327798b5-c405-45ef-8920-66f95f74bc8c admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:11.558010 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:11.605525 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:11.609340 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6c303f-ac7b-477d-aad4-9019879b7b85 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:11.615770 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb291ccb-ed3f-4041-8dd5-4e35e8fde0c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:11.616414 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb291ccb-ed3f-4041-8dd5-4e35e8fde0c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:11.616699 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb291ccb-ed3f-4041-8dd5-4e35e8fde0c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:11.616967 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb291ccb-ed3f-4041-8dd5-4e35e8fde0c4 admin admin] PATH_INFO: `/v3/users/d11d11e5e6ec4fbabad193ea95111d11` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:11.618753 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb291ccb-ed3f-4041-8dd5-4e35e8fde0c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d11d11e5e6ec4fbabad193ea95111d11)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:11.620761 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb291ccb-ed3f-4041-8dd5-4e35e8fde0c4 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:11.626018 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd71919e-633d-489e-9c90-817bd6bfa55b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:11.626705 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd71919e-633d-489e-9c90-817bd6bfa55b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:11.627023 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd71919e-633d-489e-9c90-817bd6bfa55b admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:11.627332 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd71919e-633d-489e-9c90-817bd6bfa55b admin admin] PATH_INFO: `/v3/projects/9c1ce2ef86224787a1a91bf3e8a3b778` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:11.628505 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd71919e-633d-489e-9c90-817bd6bfa55b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c1ce2ef86224787a1a91bf3e8a3b778)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:11.631723 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd71919e-633d-489e-9c90-817bd6bfa55b admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:11.828231 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-bd71919e-633d-489e-9c90-817bd6bfa55b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c1ce2ef86224787a1a91bf3e8a3b778', 'request_id': 'req-bd71919e-633d-489e-9c90-817bd6bfa55b'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:11.828443 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-fb291ccb-ed3f-4041-8dd5-4e35e8fde0c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd11d11e5e6ec4fbabad193ea95111d11'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:11.834709 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-bd71919e-633d-489e-9c90-817bd6bfa55b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c1ce2ef86224787a1a91bf3e8a3b778', 'request_id': 'req-bd71919e-633d-489e-9c90-817bd6bfa55b'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:11.840414 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-fb291ccb-ed3f-4041-8dd5-4e35e8fde0c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd11d11e5e6ec4fbabad193ea95111d11'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:11.841265 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-fb291ccb-ed3f-4041-8dd5-4e35e8fde0c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd11d11e5e6ec4fbabad193ea95111d11'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:11.922416 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-bd71919e-633d-489e-9c90-817bd6bfa55b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c1ce2ef86224787a1a91bf3e8a3b778', 'request_id': 'req-bd71919e-633d-489e-9c90-817bd6bfa55b'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:11.935551 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-bd71919e-633d-489e-9c90-817bd6bfa55b admin admin] The token cache is being invalidate because project 9c1ce2ef86224787a1a91bf3e8a3b778 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:14:11.935869 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-bd71919e-633d-489e-9c90-817bd6bfa55b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:11.936629 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-bd71919e-633d-489e-9c90-817bd6bfa55b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:11.938120 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1040/2066] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:14:11 2022] DELETE /identity/v3/projects/9c1ce2ef86224787a1a91bf3e8a3b778 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:14:11.942855 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd8bdf-d4cf-4543-8b25-d67da63f2af9 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:11.943166 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd8bdf-d4cf-4543-8b25-d67da63f2af9 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:11.943440 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd8bdf-d4cf-4543-8b25-d67da63f2af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:11.963793 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-fcdd8bdf-d4cf-4543-8b25-d67da63f2af9 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:14:11.964976 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-fcdd8bdf-d4cf-4543-8b25-d67da63f2af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:11.968373 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-fb291ccb-ed3f-4041-8dd5-4e35e8fde0c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd11d11e5e6ec4fbabad193ea95111d11'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:11.969005 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-fb291ccb-ed3f-4041-8dd5-4e35e8fde0c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd11d11e5e6ec4fbabad193ea95111d11'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:11.982280 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1027/2067] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:14:11 2022] DELETE /identity/v3/users/d11d11e5e6ec4fbabad193ea95111d11 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:14:11.986744 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8af16-4ed0-4651-ace4-38fd09c872d6 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:11.987074 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8af16-4ed0-4651-ace4-38fd09c872d6 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:11.987355 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8af16-4ed0-4651-ace4-38fd09c872d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:12.007354 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-1ef8af16-4ed0-4651-ace4-38fd09c872d6 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:14:12.008561 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-1ef8af16-4ed0-4651-ace4-38fd09c872d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:12.036686 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1041/2068] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:14:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 17:14:12.038224 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e6ba02-9620-4a15-a2de-a9fa3a727f28 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:12.038506 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e6ba02-9620-4a15-a2de-a9fa3a727f28 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:12.038779 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e6ba02-9620-4a15-a2de-a9fa3a727f28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:12.060403 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-88e6ba02-9620-4a15-a2de-a9fa3a727f28 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:14:12.061601 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-88e6ba02-9620-4a15-a2de-a9fa3a727f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:12.071402 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1028/2069] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:14:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 17:14:12.081624 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef8af16-4ed0-4651-ace4-38fd09c872d6 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:12.083023 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-1ef8af16-4ed0-4651-ace4-38fd09c872d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:12.099115 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:12.127152 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1042/2070] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:14:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 17:14:12.131396 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e6ba02-9620-4a15-a2de-a9fa3a727f28 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:12.141921 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e6ba02-9620-4a15-a2de-a9fa3a727f28 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:12.146044 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef8af16-4ed0-4651-ace4-38fd09c872d6 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:12.148088 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe8f417-0a11-4858-a674-09158c072b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:12.150104 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe8f417-0a11-4858-a674-09158c072b45 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:12.150104 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe8f417-0a11-4858-a674-09158c072b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:12.150497 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe8f417-0a11-4858-a674-09158c072b45 admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:12.150599 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063f0ea2-d1c4-4b1a-95d1-a7e09d2a1cf1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:12.151215 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f0ea2-d1c4-4b1a-95d1-a7e09d2a1cf1 service neutron] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:12.151495 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f0ea2-d1c4-4b1a-95d1-a7e09d2a1cf1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:12.151773 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f0ea2-d1c4-4b1a-95d1-a7e09d2a1cf1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:12.152162 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe8f417-0a11-4858-a674-09158c072b45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:14:12.152641 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe8f417-0a11-4858-a674-09158c072b45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:12.154401 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe8f417-0a11-4858-a674-09158c072b45 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:12.161881 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063f0ea2-d1c4-4b1a-95d1-a7e09d2a1cf1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:12.163822 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063f0ea2-d1c4-4b1a-95d1-a7e09d2a1cf1 service neutron] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:12.168824 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1043/2071] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:14:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 17:14:12.173072 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe8f417-0a11-4858-a674-09158c072b45 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:12.176174 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1029/2072] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:14:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 17:14:12.186400 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe8f417-0a11-4858-a674-09158c072b45 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:12.191295 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adacd63-7149-4939-9697-dce34c72d239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:12.191981 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adacd63-7149-4939-9697-dce34c72d239 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:12.192334 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adacd63-7149-4939-9697-dce34c72d239 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:12.193065 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adacd63-7149-4939-9697-dce34c72d239 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:12.193479 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adacd63-7149-4939-9697-dce34c72d239 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:12.195557 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adacd63-7149-4939-9697-dce34c72d239 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:12.275510 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1044/2073] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:14:12 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 17:14:12.279744 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063f0ea2-d1c4-4b1a-95d1-a7e09d2a1cf1 service neutron] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:12.291248 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063f0ea2-d1c4-4b1a-95d1-a7e09d2a1cf1 service neutron] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:12.309886 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:12.351788 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f886b0b-83b4-4106-98f4-1b38cc192693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:12.352929 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f886b0b-83b4-4106-98f4-1b38cc192693 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:12.353201 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f886b0b-83b4-4106-98f4-1b38cc192693 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:12.353477 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f886b0b-83b4-4106-98f4-1b38cc192693 admin admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:12.354094 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f886b0b-83b4-4106-98f4-1b38cc192693 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:12.356097 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f886b0b-83b4-4106-98f4-1b38cc192693 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:12.463773 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1030/2074] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:14:12 2022] POST /identity/v3/users => generated 293 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:14:12.466995 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adacd63-7149-4939-9697-dce34c72d239 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:12.478342 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adacd63-7149-4939-9697-dce34c72d239 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:12.482962 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8316aa9f-cf5d-499e-9833-dc448c8d60fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:12.483655 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8316aa9f-cf5d-499e-9833-dc448c8d60fd admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:12.483907 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8316aa9f-cf5d-499e-9833-dc448c8d60fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:12.484178 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8316aa9f-cf5d-499e-9833-dc448c8d60fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:12.484850 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8316aa9f-cf5d-499e-9833-dc448c8d60fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:12.486439 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8316aa9f-cf5d-499e-9833-dc448c8d60fd admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:12.497555 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1045/2075] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:14:12.501575 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f886b0b-83b4-4106-98f4-1b38cc192693 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:12.512290 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f886b0b-83b4-4106-98f4-1b38cc192693 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:12.517072 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb20412-a094-4d55-a433-d460da4e200d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:12.517703 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb20412-a094-4d55-a433-d460da4e200d admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:12.517989 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb20412-a094-4d55-a433-d460da4e200d admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:12.518266 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb20412-a094-4d55-a433-d460da4e200d admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:12.518922 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb20412-a094-4d55-a433-d460da4e200d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:12.520551 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb20412-a094-4d55-a433-d460da4e200d admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:12.534611 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1031/2076] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:14:12.542789 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8316aa9f-cf5d-499e-9833-dc448c8d60fd admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:12.560057 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8316aa9f-cf5d-499e-9833-dc448c8d60fd admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:12.564845 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ac0753-60d2-4ba4-9bd6-0f2f32c9f890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:12.565505 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ac0753-60d2-4ba4-9bd6-0f2f32c9f890 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:12.565787 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ac0753-60d2-4ba4-9bd6-0f2f32c9f890 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:12.566061 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ac0753-60d2-4ba4-9bd6-0f2f32c9f890 admin admin] PATH_INFO: `/v3/projects/d0c36f74e87a43aa80fe441dd9f01ada/users/27af8c8e6368464fbf990459508eb700/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:12.581697 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ac0753-60d2-4ba4-9bd6-0f2f32c9f890 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0c36f74e87a43aa80fe441dd9f01ada, user_id=27af8c8e6368464fbf990459508eb700, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:12.584361 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ac0753-60d2-4ba4-9bd6-0f2f32c9f890 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:12.663519 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1046/2077] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:14:12 2022] PUT /identity/v3/projects/d0c36f74e87a43aa80fe441dd9f01ada/users/27af8c8e6368464fbf990459508eb700/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:14:12.676636 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb20412-a094-4d55-a433-d460da4e200d admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:12.678676 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-edb20412-a094-4d55-a433-d460da4e200d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:12.678839 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e40432-098e-44ae-a30e-ef36463c4b20 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:12.679019 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e40432-098e-44ae-a30e-ef36463c4b20 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:12.679452 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e40432-098e-44ae-a30e-ef36463c4b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:12.699461 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb20412-a094-4d55-a433-d460da4e200d admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:12.710647 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-36e40432-098e-44ae-a30e-ef36463c4b20 None None] MFA Rules not processed for user `27af8c8e6368464fbf990459508eb700`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:14:12.713667 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-36e40432-098e-44ae-a30e-ef36463c4b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:12.767354 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d90dd4d-f570-4489-b47a-6f0d6d87b4c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:12.768314 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90dd4d-f570-4489-b47a-6f0d6d87b4c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:12.768797 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90dd4d-f570-4489-b47a-6f0d6d87b4c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:12.769140 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90dd4d-f570-4489-b47a-6f0d6d87b4c5 admin admin] PATH_INFO: `/v3/projects/76a004cabf7441f985ea431720a1666d` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:12.770675 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d90dd4d-f570-4489-b47a-6f0d6d87b4c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76a004cabf7441f985ea431720a1666d)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:12.773527 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d90dd4d-f570-4489-b47a-6f0d6d87b4c5 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:12.798493 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1047/2078] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:14:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:14:12.803418 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e40432-098e-44ae-a30e-ef36463c4b20 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:12.813900 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e40432-098e-44ae-a30e-ef36463c4b20 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:12.819422 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5759faa-e3a5-487d-9f5e-9cdd8a396460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:12.820025 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5759faa-e3a5-487d-9f5e-9cdd8a396460 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:12.820858 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5759faa-e3a5-487d-9f5e-9cdd8a396460 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:12.821138 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5759faa-e3a5-487d-9f5e-9cdd8a396460 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:12.821761 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5759faa-e3a5-487d-9f5e-9cdd8a396460 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:12.823715 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5759faa-e3a5-487d-9f5e-9cdd8a396460 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:12.879888 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-1d90dd4d-f570-4489-b47a-6f0d6d87b4c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76a004cabf7441f985ea431720a1666d', 'request_id': 'req-1d90dd4d-f570-4489-b47a-6f0d6d87b4c5'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:12.915361 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1048/2079] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:14:12 2022] POST /identity/v3/projects => generated 387 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 17:14:12.919672 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5759faa-e3a5-487d-9f5e-9cdd8a396460 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:12.958669 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:12.962994 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-1d90dd4d-f570-4489-b47a-6f0d6d87b4c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76a004cabf7441f985ea431720a1666d', 'request_id': 'req-1d90dd4d-f570-4489-b47a-6f0d6d87b4c5'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:12.969343 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-1d90dd4d-f570-4489-b47a-6f0d6d87b4c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76a004cabf7441f985ea431720a1666d', 'request_id': 'req-1d90dd4d-f570-4489-b47a-6f0d6d87b4c5'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:12.980994 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-1d90dd4d-f570-4489-b47a-6f0d6d87b4c5 admin admin] The token cache is being invalidate because project 76a004cabf7441f985ea431720a1666d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81650) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:14:12.981286 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-1d90dd4d-f570-4489-b47a-6f0d6d87b4c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:12.981581 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5759faa-e3a5-487d-9f5e-9cdd8a396460 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:12.981945 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-1d90dd4d-f570-4489-b47a-6f0d6d87b4c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:12.983572 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1032/2080] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:14:12 2022] DELETE /identity/v3/projects/76a004cabf7441f985ea431720a1666d => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:14:12.987398 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d90dd4d-f570-4489-b47a-6f0d6d87b4c5 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:12.987518 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247d5e49-4dcb-4549-94a6-1439b26c0816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:12.988150 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247d5e49-4dcb-4549-94a6-1439b26c0816 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:12.988511 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247d5e49-4dcb-4549-94a6-1439b26c0816 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:12.988703 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-1d90dd4d-f570-4489-b47a-6f0d6d87b4c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:12.988794 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247d5e49-4dcb-4549-94a6-1439b26c0816 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:12.989476 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247d5e49-4dcb-4549-94a6-1439b26c0816 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:12.991489 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247d5e49-4dcb-4549-94a6-1439b26c0816 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:13.021073 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:13.053496 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d90dd4d-f570-4489-b47a-6f0d6d87b4c5 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:13.058751 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fc99f1-4ff5-4e0f-b9b8-4b34da6e44f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:13.059422 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc99f1-4ff5-4e0f-b9b8-4b34da6e44f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:13.059708 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc99f1-4ff5-4e0f-b9b8-4b34da6e44f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:13.059985 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc99f1-4ff5-4e0f-b9b8-4b34da6e44f2 admin admin] PATH_INFO: `/v3/users/c608540c7d494713bf9574f18dcfdff1` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:13.061104 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fc99f1-4ff5-4e0f-b9b8-4b34da6e44f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c608540c7d494713bf9574f18dcfdff1)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:13.063098 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fc99f1-4ff5-4e0f-b9b8-4b34da6e44f2 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:13.071642 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1049/2081] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:14:12 2022] POST /identity/v3/users => generated 300 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 17:14:13.075753 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247d5e49-4dcb-4549-94a6-1439b26c0816 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:13.077368 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-247d5e49-4dcb-4549-94a6-1439b26c0816 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:13.089227 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247d5e49-4dcb-4549-94a6-1439b26c0816 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:13.093893 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a350b4-49f9-4679-96cb-1414eee8677f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:13.094485 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a350b4-49f9-4679-96cb-1414eee8677f admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:13.094762 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a350b4-49f9-4679-96cb-1414eee8677f admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:13.095060 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a350b4-49f9-4679-96cb-1414eee8677f admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:13.095689 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a350b4-49f9-4679-96cb-1414eee8677f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:13.097277 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a350b4-49f9-4679-96cb-1414eee8677f admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:13.107293 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1050/2082] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:14:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:14:13.111346 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a350b4-49f9-4679-96cb-1414eee8677f admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:13.121391 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a350b4-49f9-4679-96cb-1414eee8677f admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:13.126141 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b146e264-01a2-4b99-b92a-c66cc5b9a039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:13.126754 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b146e264-01a2-4b99-b92a-c66cc5b9a039 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:13.127107 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b146e264-01a2-4b99-b92a-c66cc5b9a039 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:13.127388 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b146e264-01a2-4b99-b92a-c66cc5b9a039 admin admin] PATH_INFO: `/v3/projects/8450f5fcda4b43bb9f87ad95d0875218/users/16c8b44c5fe446f28ac8dce3f97578d0/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:13.142412 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b146e264-01a2-4b99-b92a-c66cc5b9a039 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8450f5fcda4b43bb9f87ad95d0875218, user_id=16c8b44c5fe446f28ac8dce3f97578d0, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:13.145075 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b146e264-01a2-4b99-b92a-c66cc5b9a039 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:13.234654 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1051/2083] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:14:13 2022] PUT /identity/v3/projects/8450f5fcda4b43bb9f87ad95d0875218/users/16c8b44c5fe446f28ac8dce3f97578d0/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:14:13.238598 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b146e264-01a2-4b99-b92a-c66cc5b9a039 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:13.264734 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-b4fc99f1-4ff5-4e0f-b9b8-4b34da6e44f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c608540c7d494713bf9574f18dcfdff1'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:13.276895 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-b4fc99f1-4ff5-4e0f-b9b8-4b34da6e44f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c608540c7d494713bf9574f18dcfdff1'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:13.277644 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-b4fc99f1-4ff5-4e0f-b9b8-4b34da6e44f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c608540c7d494713bf9574f18dcfdff1'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:13.308936 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b146e264-01a2-4b99-b92a-c66cc5b9a039 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:13.310602 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaadb656-166f-4896-8be3-5247afcc4ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:13.311242 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaadb656-166f-4896-8be3-5247afcc4ee3 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:13.311555 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaadb656-166f-4896-8be3-5247afcc4ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:13.311828 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaadb656-166f-4896-8be3-5247afcc4ee3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:13.312497 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaadb656-166f-4896-8be3-5247afcc4ee3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:13.314103 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaadb656-166f-4896-8be3-5247afcc4ee3 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:13.324711 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1052/2084] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:14:13 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 17:14:13.328856 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaadb656-166f-4896-8be3-5247afcc4ee3 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:13.340742 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaadb656-166f-4896-8be3-5247afcc4ee3 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:13.343309 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8d7367-3813-4fdb-a895-64622d140c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:13.343972 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8d7367-3813-4fdb-a895-64622d140c40 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:13.344248 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8d7367-3813-4fdb-a895-64622d140c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:13.344519 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8d7367-3813-4fdb-a895-64622d140c40 admin admin] PATH_INFO: `/v3/projects/8450f5fcda4b43bb9f87ad95d0875218/users/16c8b44c5fe446f28ac8dce3f97578d0/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:13.346414 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8d7367-3813-4fdb-a895-64622d140c40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8450f5fcda4b43bb9f87ad95d0875218, user_id=16c8b44c5fe446f28ac8dce3f97578d0, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:13.349091 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8d7367-3813-4fdb-a895-64622d140c40 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:13.359331 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-b4fc99f1-4ff5-4e0f-b9b8-4b34da6e44f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c608540c7d494713bf9574f18dcfdff1'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:13.359985 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-b4fc99f1-4ff5-4e0f-b9b8-4b34da6e44f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c608540c7d494713bf9574f18dcfdff1'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:13.360288 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1053/2085] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:14:13 2022] PUT /identity/v3/projects/8450f5fcda4b43bb9f87ad95d0875218/users/16c8b44c5fe446f28ac8dce3f97578d0/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:14:13.372030 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5c026-c693-4cd3-90bd-483552f05fcc None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:13.372401 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5c026-c693-4cd3-90bd-483552f05fcc None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:13.372788 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5c026-c693-4cd3-90bd-483552f05fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:13.374266 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1033/2086] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:14:12 2022] DELETE /identity/v3/users/c608540c7d494713bf9574f18dcfdff1 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:14:13.405621 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-2ed5c026-c693-4cd3-90bd-483552f05fcc None None] MFA Rules not processed for user `16c8b44c5fe446f28ac8dce3f97578d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:14:13.407038 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-2ed5c026-c693-4cd3-90bd-483552f05fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:13.486143 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:13.491745 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1054/2087] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:14:13 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:14:13.504381 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fc99f1-4ff5-4e0f-b9b8-4b34da6e44f2 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:13.505524 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-b4fc99f1-4ff5-4e0f-b9b8-4b34da6e44f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:13.567861 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fc99f1-4ff5-4e0f-b9b8-4b34da6e44f2 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:13.583470 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3a7002-41e5-41ab-a0d9-d843e4d5454b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:13.583470 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a7002-41e5-41ab-a0d9-d843e4d5454b service cinder] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:13.583470 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a7002-41e5-41ab-a0d9-d843e4d5454b service cinder] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:13.583470 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a7002-41e5-41ab-a0d9-d843e4d5454b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:13.588789 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3a7002-41e5-41ab-a0d9-d843e4d5454b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:13.590364 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3a7002-41e5-41ab-a0d9-d843e4d5454b service cinder] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:13.604379 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1034/2088] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:14:13 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:14:13.816718 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed5c026-c693-4cd3-90bd-483552f05fcc None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:13.818505 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-2ed5c026-c693-4cd3-90bd-483552f05fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:13.890728 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed5c026-c693-4cd3-90bd-483552f05fcc None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:13.896229 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d840647-ae7c-4a46-91ac-391a03c77106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:13.901773 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d840647-ae7c-4a46-91ac-391a03c77106 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:13.901773 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d840647-ae7c-4a46-91ac-391a03c77106 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:13.901773 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d840647-ae7c-4a46-91ac-391a03c77106 admin admin] PATH_INFO: `/v3/projects/f84b4b0ea2fc474c909daef8f8badd45` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:13.901773 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d840647-ae7c-4a46-91ac-391a03c77106 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f84b4b0ea2fc474c909daef8f8badd45)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:13.901773 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d840647-ae7c-4a46-91ac-391a03c77106 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:14.052053 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-9d840647-ae7c-4a46-91ac-391a03c77106 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f84b4b0ea2fc474c909daef8f8badd45', 'request_id': 'req-9d840647-ae7c-4a46-91ac-391a03c77106'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:14.058451 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-9d840647-ae7c-4a46-91ac-391a03c77106 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f84b4b0ea2fc474c909daef8f8badd45', 'request_id': 'req-9d840647-ae7c-4a46-91ac-391a03c77106'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:14.141725 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-9d840647-ae7c-4a46-91ac-391a03c77106 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f84b4b0ea2fc474c909daef8f8badd45', 'request_id': 'req-9d840647-ae7c-4a46-91ac-391a03c77106'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:14.153671 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-9d840647-ae7c-4a46-91ac-391a03c77106 admin admin] The token cache is being invalidate because project f84b4b0ea2fc474c909daef8f8badd45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:14:14.153961 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-9d840647-ae7c-4a46-91ac-391a03c77106 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:14.154588 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-9d840647-ae7c-4a46-91ac-391a03c77106 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:14.156950 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1055/2089] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:14:13 2022] DELETE /identity/v3/projects/f84b4b0ea2fc474c909daef8f8badd45 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:14:14.161852 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28cd14f-ddb1-4a00-ae19-ea31d83296b4 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:14.162344 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28cd14f-ddb1-4a00-ae19-ea31d83296b4 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:14.162434 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28cd14f-ddb1-4a00-ae19-ea31d83296b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:14.183644 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-c28cd14f-ddb1-4a00-ae19-ea31d83296b4 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:14:14.184854 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-c28cd14f-ddb1-4a00-ae19-ea31d83296b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:14.234670 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:14.252060 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1035/2090] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:14:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:14:14.260919 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250223d9-c8c6-4e92-979d-c11ef1acda80 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:14.261213 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250223d9-c8c6-4e92-979d-c11ef1acda80 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:14.261500 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250223d9-c8c6-4e92-979d-c11ef1acda80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:14.295857 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-250223d9-c8c6-4e92-979d-c11ef1acda80 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:14:14.297122 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-250223d9-c8c6-4e92-979d-c11ef1acda80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:14.305387 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1056/2091] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:14:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:14:14.313892 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28cd14f-ddb1-4a00-ae19-ea31d83296b4 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:14.324740 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28cd14f-ddb1-4a00-ae19-ea31d83296b4 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:14.330142 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f793820-64d9-4598-83ef-d68d8fa10c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:14.332665 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f793820-64d9-4598-83ef-d68d8fa10c50 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:14.332665 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f793820-64d9-4598-83ef-d68d8fa10c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:14.332665 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f793820-64d9-4598-83ef-d68d8fa10c50 admin admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:14.332665 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f793820-64d9-4598-83ef-d68d8fa10c50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:14:14.332665 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f793820-64d9-4598-83ef-d68d8fa10c50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:14.333965 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f793820-64d9-4598-83ef-d68d8fa10c50 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:14.346109 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1036/2092] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:14:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 17:14:14.350478 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250223d9-c8c6-4e92-979d-c11ef1acda80 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:14.359917 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250223d9-c8c6-4e92-979d-c11ef1acda80 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:14.364997 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0384255-b60f-425c-9f01-f327eed64631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:14.365601 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0384255-b60f-425c-9f01-f327eed64631 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:14.365876 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0384255-b60f-425c-9f01-f327eed64631 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:14.366146 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0384255-b60f-425c-9f01-f327eed64631 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:14.366822 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0384255-b60f-425c-9f01-f327eed64631 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:14.370841 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0384255-b60f-425c-9f01-f327eed64631 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:14.463871 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1057/2093] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:14:14 2022] POST /identity/v3/projects => generated 379 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 17:14:14.470255 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f793820-64d9-4598-83ef-d68d8fa10c50 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:14.480823 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f793820-64d9-4598-83ef-d68d8fa10c50 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:14.540482 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852ee04f-192a-4db3-84c5-6bd917beffbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:14.541406 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ee04f-192a-4db3-84c5-6bd917beffbc admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:14.541487 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ee04f-192a-4db3-84c5-6bd917beffbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:14.541701 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ee04f-192a-4db3-84c5-6bd917beffbc admin admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:14.542689 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852ee04f-192a-4db3-84c5-6bd917beffbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:14.545936 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852ee04f-192a-4db3-84c5-6bd917beffbc admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:14.620744 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1037/2094] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:14:14 2022] POST /identity/v3/users => generated 290 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 17:14:14.633593 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0384255-b60f-425c-9f01-f327eed64631 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:14.642556 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0384255-b60f-425c-9f01-f327eed64631 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:14.647583 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226349c7-b1fd-4548-b8fa-9523e891a26d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:14.648229 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226349c7-b1fd-4548-b8fa-9523e891a26d admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:14.648514 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226349c7-b1fd-4548-b8fa-9523e891a26d admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:14.648821 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226349c7-b1fd-4548-b8fa-9523e891a26d admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:14.649437 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226349c7-b1fd-4548-b8fa-9523e891a26d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:14.651079 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226349c7-b1fd-4548-b8fa-9523e891a26d admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:14.663014 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1058/2095] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:14:14.681260 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852ee04f-192a-4db3-84c5-6bd917beffbc admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:14.691905 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852ee04f-192a-4db3-84c5-6bd917beffbc admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:14.697983 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9405de-5f44-4244-aead-f3e97c304777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:14.698538 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9405de-5f44-4244-aead-f3e97c304777 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:14.698873 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9405de-5f44-4244-aead-f3e97c304777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:14.699177 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9405de-5f44-4244-aead-f3e97c304777 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:14.699810 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9405de-5f44-4244-aead-f3e97c304777 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:14.701409 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9405de-5f44-4244-aead-f3e97c304777 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:14.713854 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1038/2096] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 17:14:14.718078 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226349c7-b1fd-4548-b8fa-9523e891a26d admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:14.728837 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226349c7-b1fd-4548-b8fa-9523e891a26d admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:14.733638 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94012276-202d-4ca5-a04a-89d18e5738e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:14.734299 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94012276-202d-4ca5-a04a-89d18e5738e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:14.734657 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94012276-202d-4ca5-a04a-89d18e5738e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:14.734861 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94012276-202d-4ca5-a04a-89d18e5738e2 admin admin] PATH_INFO: `/v3/projects/ee1edd3e6d4e42a18f345c846a484b73/users/1542f734d249496e90d8278a95d285a3/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:14.758718 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94012276-202d-4ca5-a04a-89d18e5738e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee1edd3e6d4e42a18f345c846a484b73, user_id=1542f734d249496e90d8278a95d285a3, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:14.762135 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94012276-202d-4ca5-a04a-89d18e5738e2 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:14.836773 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9405de-5f44-4244-aead-f3e97c304777 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:14.838498 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-db9405de-5f44-4244-aead-f3e97c304777 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:14.852758 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1059/2097] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:14:14 2022] PUT /identity/v3/projects/ee1edd3e6d4e42a18f345c846a484b73/users/1542f734d249496e90d8278a95d285a3/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:14:14.854023 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9405de-5f44-4244-aead-f3e97c304777 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:14.864156 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c0b028-ecc4-439d-b885-3c92c46e8a93 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:14.864520 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c0b028-ecc4-439d-b885-3c92c46e8a93 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:14.864917 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c0b028-ecc4-439d-b885-3c92c46e8a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:14.907208 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-91c0b028-ecc4-439d-b885-3c92c46e8a93 None None] MFA Rules not processed for user `1542f734d249496e90d8278a95d285a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:14:14.908419 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-91c0b028-ecc4-439d-b885-3c92c46e8a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:14.921322 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f449bfa2-ebc4-472c-b17b-0f861a43b6ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:14.921524 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:14.921965 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f449bfa2-ebc4-472c-b17b-0f861a43b6ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:14.922251 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f449bfa2-ebc4-472c-b17b-0f861a43b6ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:14.922537 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f449bfa2-ebc4-472c-b17b-0f861a43b6ed admin admin] PATH_INFO: `/v3/users/27af8c8e6368464fbf990459508eb700` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:14.923744 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f449bfa2-ebc4-472c-b17b-0f861a43b6ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=27af8c8e6368464fbf990459508eb700)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:14.930331 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f449bfa2-ebc4-472c-b17b-0f861a43b6ed admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:14.973867 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1060/2098] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:14:14 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:14:14.979201 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c0b028-ecc4-439d-b885-3c92c46e8a93 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:14.989117 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c0b028-ecc4-439d-b885-3c92c46e8a93 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:14.994481 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75964e0c-f365-4614-99ca-79094bf1d0b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:14.995097 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75964e0c-f365-4614-99ca-79094bf1d0b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:14.995406 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75964e0c-f365-4614-99ca-79094bf1d0b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:14.995684 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75964e0c-f365-4614-99ca-79094bf1d0b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:14.996296 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75964e0c-f365-4614-99ca-79094bf1d0b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:14.998187 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75964e0c-f365-4614-99ca-79094bf1d0b5 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:15.101361 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1061/2099] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:14:14 2022] POST /identity/v3/projects => generated 377 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 17:14:15.106171 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75964e0c-f365-4614-99ca-79094bf1d0b5 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:15.136174 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-f449bfa2-ebc4-472c-b17b-0f861a43b6ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27af8c8e6368464fbf990459508eb700'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:15.144278 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:15.149339 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-f449bfa2-ebc4-472c-b17b-0f861a43b6ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27af8c8e6368464fbf990459508eb700'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:15.153949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-f449bfa2-ebc4-472c-b17b-0f861a43b6ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27af8c8e6368464fbf990459508eb700'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:15.169671 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75964e0c-f365-4614-99ca-79094bf1d0b5 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:15.174390 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88edb291-fa21-4c51-bcac-6da821ccc3de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:15.175026 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edb291-fa21-4c51-bcac-6da821ccc3de admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:15.175306 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edb291-fa21-4c51-bcac-6da821ccc3de admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:15.175576 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edb291-fa21-4c51-bcac-6da821ccc3de admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:15.176215 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88edb291-fa21-4c51-bcac-6da821ccc3de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:15.178110 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88edb291-fa21-4c51-bcac-6da821ccc3de admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:15.254068 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-f449bfa2-ebc4-472c-b17b-0f861a43b6ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27af8c8e6368464fbf990459508eb700'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:15.255115 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-f449bfa2-ebc4-472c-b17b-0f861a43b6ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27af8c8e6368464fbf990459508eb700'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:15.259822 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1062/2100] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:14:15 2022] POST /identity/v3/users => generated 289 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 17:14:15.265386 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88edb291-fa21-4c51-bcac-6da821ccc3de admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:15.267370 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-88edb291-fa21-4c51-bcac-6da821ccc3de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:15.271875 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1039/2101] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:14:14 2022] DELETE /identity/v3/users/27af8c8e6368464fbf990459508eb700 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:14:15.277152 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed12ce53-0930-4263-8b50-89f72ea20500 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:15.277581 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed12ce53-0930-4263-8b50-89f72ea20500 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:15.277969 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed12ce53-0930-4263-8b50-89f72ea20500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:15.280484 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88edb291-fa21-4c51-bcac-6da821ccc3de admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:15.303220 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-ed12ce53-0930-4263-8b50-89f72ea20500 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:14:15.304714 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-ed12ce53-0930-4263-8b50-89f72ea20500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:15.356313 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870b72f5-9b5e-4453-bc80-15d422e40923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:15.356868 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870b72f5-9b5e-4453-bc80-15d422e40923 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:15.357153 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870b72f5-9b5e-4453-bc80-15d422e40923 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:15.357489 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870b72f5-9b5e-4453-bc80-15d422e40923 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:15.358137 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870b72f5-9b5e-4453-bc80-15d422e40923 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:15.359923 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870b72f5-9b5e-4453-bc80-15d422e40923 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:15.370291 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1063/2102] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 17:14:15.376144 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870b72f5-9b5e-4453-bc80-15d422e40923 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:15.376269 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1040/2103] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:14:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:14:15.386353 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870b72f5-9b5e-4453-bc80-15d422e40923 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:15.386946 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed12ce53-0930-4263-8b50-89f72ea20500 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:15.388370 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-ed12ce53-0930-4263-8b50-89f72ea20500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:15.391169 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25889798-12b0-4505-82d3-769df510f683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:15.391767 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25889798-12b0-4505-82d3-769df510f683 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:15.392044 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25889798-12b0-4505-82d3-769df510f683 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:15.392317 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25889798-12b0-4505-82d3-769df510f683 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:15.392984 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25889798-12b0-4505-82d3-769df510f683 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:15.394592 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25889798-12b0-4505-82d3-769df510f683 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:15.406199 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1064/2104] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 17:14:15.410280 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25889798-12b0-4505-82d3-769df510f683 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:15.420599 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25889798-12b0-4505-82d3-769df510f683 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:15.425394 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b2b7bc-a691-42cf-968b-f8cdd038087f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:15.426028 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2b7bc-a691-42cf-968b-f8cdd038087f admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:15.426309 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2b7bc-a691-42cf-968b-f8cdd038087f admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:15.426589 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2b7bc-a691-42cf-968b-f8cdd038087f admin admin] PATH_INFO: `/v3/projects/41bb531a07744267a28589bae4907f4e/users/b186d23bf56f4aacacd9f10d17391079/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:15.428795 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:15.442513 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b2b7bc-a691-42cf-968b-f8cdd038087f admin admin] RBAC: Authorizing `identity:create_grant(project_id=41bb531a07744267a28589bae4907f4e, user_id=b186d23bf56f4aacacd9f10d17391079, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:15.445273 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b2b7bc-a691-42cf-968b-f8cdd038087f admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:15.447152 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed12ce53-0930-4263-8b50-89f72ea20500 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:15.450803 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac97e4cc-a5f1-423d-a190-f6afad2992f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:15.451446 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac97e4cc-a5f1-423d-a190-f6afad2992f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:15.451727 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac97e4cc-a5f1-423d-a190-f6afad2992f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:15.451999 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac97e4cc-a5f1-423d-a190-f6afad2992f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:15.461705 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac97e4cc-a5f1-423d-a190-f6afad2992f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:15.463177 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac97e4cc-a5f1-423d-a190-f6afad2992f3 service neutron] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:15.472418 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1041/2105] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:14:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 17:14:15.519683 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1065/2106] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:14:15 2022] PUT /identity/v3/projects/41bb531a07744267a28589bae4907f4e/users/b186d23bf56f4aacacd9f10d17391079/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:14:15.530540 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e2dd5d-8c55-43d4-898f-36b9d1e88917 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:15.530993 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e2dd5d-8c55-43d4-898f-36b9d1e88917 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:15.531133 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e2dd5d-8c55-43d4-898f-36b9d1e88917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:15.559515 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-74e2dd5d-8c55-43d4-898f-36b9d1e88917 None None] MFA Rules not processed for user `b186d23bf56f4aacacd9f10d17391079`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:14:15.560727 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-74e2dd5d-8c55-43d4-898f-36b9d1e88917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:15.624697 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1042/2107] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:14:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:14:15.629123 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b2b7bc-a691-42cf-968b-f8cdd038087f admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:15.694422 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b2b7bc-a691-42cf-968b-f8cdd038087f admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:15.700324 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de32f0c-fd1b-456a-b35b-25ec4793b9be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:15.700936 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de32f0c-fd1b-456a-b35b-25ec4793b9be admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:15.701217 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de32f0c-fd1b-456a-b35b-25ec4793b9be admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:15.701489 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de32f0c-fd1b-456a-b35b-25ec4793b9be admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:15.702213 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de32f0c-fd1b-456a-b35b-25ec4793b9be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:15.704212 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de32f0c-fd1b-456a-b35b-25ec4793b9be admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:15.798615 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1066/2108] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:14:15 2022] POST /identity/v3/projects => generated 379 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:14:15.802640 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e2dd5d-8c55-43d4-898f-36b9d1e88917 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:15.813799 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e2dd5d-8c55-43d4-898f-36b9d1e88917 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:15.818452 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:15.880103 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de32f0c-fd1b-456a-b35b-25ec4793b9be admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:15.881999 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-8de32f0c-fd1b-456a-b35b-25ec4793b9be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:15.890192 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71b31ca-f815-412a-9dbf-624f654b6e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:15.891338 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b31ca-f815-412a-9dbf-624f654b6e61 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:15.891730 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b31ca-f815-412a-9dbf-624f654b6e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:15.892269 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b31ca-f815-412a-9dbf-624f654b6e61 admin admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:15.893007 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71b31ca-f815-412a-9dbf-624f654b6e61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:15.899034 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71b31ca-f815-412a-9dbf-624f654b6e61 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:15.981364 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de32f0c-fd1b-456a-b35b-25ec4793b9be admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:15.986032 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0735644-9867-4eff-9399-286b053b6868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:15.986656 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0735644-9867-4eff-9399-286b053b6868 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:15.986969 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0735644-9867-4eff-9399-286b053b6868 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:15.987264 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0735644-9867-4eff-9399-286b053b6868 admin admin] PATH_INFO: `/v3/projects/d0c36f74e87a43aa80fe441dd9f01ada` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:15.988365 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0735644-9867-4eff-9399-286b053b6868 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0c36f74e87a43aa80fe441dd9f01ada)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:15.990261 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0735644-9867-4eff-9399-286b053b6868 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:15.999960 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1043/2109] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:14:15 2022] POST /identity/v3/users => generated 296 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:14:16.003716 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71b31ca-f815-412a-9dbf-624f654b6e61 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:16.014179 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71b31ca-f815-412a-9dbf-624f654b6e61 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:16.018769 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8e0283-944e-4d23-bc3e-5c6c416a92b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:16.019401 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8e0283-944e-4d23-bc3e-5c6c416a92b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:16.019684 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8e0283-944e-4d23-bc3e-5c6c416a92b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:16.019959 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8e0283-944e-4d23-bc3e-5c6c416a92b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:16.020600 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8e0283-944e-4d23-bc3e-5c6c416a92b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:16.022559 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8e0283-944e-4d23-bc3e-5c6c416a92b4 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:16.032979 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1044/2110] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:14:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:14:16.037037 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8e0283-944e-4d23-bc3e-5c6c416a92b4 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:16.046823 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8e0283-944e-4d23-bc3e-5c6c416a92b4 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:16.051557 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d9613e-feb4-4b63-a6e2-55d5df8920b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:16.052186 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d9613e-feb4-4b63-a6e2-55d5df8920b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:16.054332 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d9613e-feb4-4b63-a6e2-55d5df8920b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:16.054332 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d9613e-feb4-4b63-a6e2-55d5df8920b1 admin admin] PATH_INFO: `/v3/projects/d8d68a86022a41928a421d4cbeb33b31/users/7ad8931447964e1a9fcdf5b8238910ad/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:16.067562 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d9613e-feb4-4b63-a6e2-55d5df8920b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8d68a86022a41928a421d4cbeb33b31, user_id=7ad8931447964e1a9fcdf5b8238910ad, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:16.070831 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d9613e-feb4-4b63-a6e2-55d5df8920b1 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:16.153749 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1045/2111] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:14:16 2022] PUT /identity/v3/projects/d8d68a86022a41928a421d4cbeb33b31/users/7ad8931447964e1a9fcdf5b8238910ad/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:14:16.157606 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d9613e-feb4-4b63-a6e2-55d5df8920b1 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:16.162089 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-d0735644-9867-4eff-9399-286b053b6868 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0c36f74e87a43aa80fe441dd9f01ada', 'request_id': 'req-d0735644-9867-4eff-9399-286b053b6868'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:16.170447 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-d0735644-9867-4eff-9399-286b053b6868 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0c36f74e87a43aa80fe441dd9f01ada', 'request_id': 'req-d0735644-9867-4eff-9399-286b053b6868'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:16.220847 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d9613e-feb4-4b63-a6e2-55d5df8920b1 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:16.259817 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-d0735644-9867-4eff-9399-286b053b6868 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0c36f74e87a43aa80fe441dd9f01ada', 'request_id': 'req-d0735644-9867-4eff-9399-286b053b6868'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:16.267113 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:16.271461 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-d0735644-9867-4eff-9399-286b053b6868 admin admin] The token cache is being invalidate because project d0c36f74e87a43aa80fe441dd9f01ada was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:14:16.271840 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-d0735644-9867-4eff-9399-286b053b6868 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:16.272501 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-d0735644-9867-4eff-9399-286b053b6868 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:16.274069 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1067/2112] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:14:15 2022] DELETE /identity/v3/projects/d0c36f74e87a43aa80fe441dd9f01ada => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:14:16.277908 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0735644-9867-4eff-9399-286b053b6868 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:16.278476 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d138120-d45e-4fbe-a002-e30400a85ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:16.279130 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d138120-d45e-4fbe-a002-e30400a85ca9 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:16.279254 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-d0735644-9867-4eff-9399-286b053b6868 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:16.279415 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d138120-d45e-4fbe-a002-e30400a85ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:16.279692 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d138120-d45e-4fbe-a002-e30400a85ca9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:16.280362 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d138120-d45e-4fbe-a002-e30400a85ca9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:16.282010 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d138120-d45e-4fbe-a002-e30400a85ca9 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:16.293098 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0735644-9867-4eff-9399-286b053b6868 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:16.293532 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1046/2113] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:14:16 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 17:14:16.297931 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d493ec2e-a320-464a-8348-9c5e871b7381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:16.298558 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493ec2e-a320-464a-8348-9c5e871b7381 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:16.298839 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493ec2e-a320-464a-8348-9c5e871b7381 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:16.299209 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493ec2e-a320-464a-8348-9c5e871b7381 admin admin] PATH_INFO: `/v3/users/16c8b44c5fe446f28ac8dce3f97578d0` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:16.300348 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d493ec2e-a320-464a-8348-9c5e871b7381 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16c8b44c5fe446f28ac8dce3f97578d0)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:16.301952 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d138120-d45e-4fbe-a002-e30400a85ca9 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:16.302286 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d493ec2e-a320-464a-8348-9c5e871b7381 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:16.303775 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-4d138120-d45e-4fbe-a002-e30400a85ca9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:16.317548 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d138120-d45e-4fbe-a002-e30400a85ca9 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:16.322406 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca27b52-7200-49ba-a530-ba656c0a85e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:16.323088 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca27b52-7200-49ba-a530-ba656c0a85e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:16.323370 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca27b52-7200-49ba-a530-ba656c0a85e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:16.323797 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca27b52-7200-49ba-a530-ba656c0a85e2 admin admin] PATH_INFO: `/v3/projects/d8d68a86022a41928a421d4cbeb33b31/users/7ad8931447964e1a9fcdf5b8238910ad/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:16.325703 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca27b52-7200-49ba-a530-ba656c0a85e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8d68a86022a41928a421d4cbeb33b31, user_id=7ad8931447964e1a9fcdf5b8238910ad, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:16.328384 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca27b52-7200-49ba-a530-ba656c0a85e2 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:16.330678 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:16.341801 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1047/2114] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:14:16 2022] PUT /identity/v3/projects/d8d68a86022a41928a421d4cbeb33b31/users/7ad8931447964e1a9fcdf5b8238910ad/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:14:16.352418 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030abcb3-d123-405e-a843-5a3f79df9f07 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:16.352672 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030abcb3-d123-405e-a843-5a3f79df9f07 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:16.352947 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030abcb3-d123-405e-a843-5a3f79df9f07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:16.380896 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-030abcb3-d123-405e-a843-5a3f79df9f07 None None] MFA Rules not processed for user `7ad8931447964e1a9fcdf5b8238910ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:14:16.382142 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-030abcb3-d123-405e-a843-5a3f79df9f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:16.451331 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1048/2115] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:14:16 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 17:14:16.462967 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030abcb3-d123-405e-a843-5a3f79df9f07 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:16.464471 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-030abcb3-d123-405e-a843-5a3f79df9f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:16.485874 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-d493ec2e-a320-464a-8348-9c5e871b7381 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16c8b44c5fe446f28ac8dce3f97578d0'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:16.520933 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030abcb3-d123-405e-a843-5a3f79df9f07 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:16.524961 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c099b9-8850-4a48-892c-3dc4b35bf32b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:16.525593 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c099b9-8850-4a48-892c-3dc4b35bf32b service cinder] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:16.525874 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c099b9-8850-4a48-892c-3dc4b35bf32b service cinder] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:16.526287 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c099b9-8850-4a48-892c-3dc4b35bf32b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:16.536048 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c099b9-8850-4a48-892c-3dc4b35bf32b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:16.537601 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c099b9-8850-4a48-892c-3dc4b35bf32b service cinder] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:16.547881 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1049/2116] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:14:16 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 17:14:16.624867 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-d493ec2e-a320-464a-8348-9c5e871b7381 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16c8b44c5fe446f28ac8dce3f97578d0'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:16.625525 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-d493ec2e-a320-464a-8348-9c5e871b7381 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16c8b44c5fe446f28ac8dce3f97578d0'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:16.626185 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-d493ec2e-a320-464a-8348-9c5e871b7381 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16c8b44c5fe446f28ac8dce3f97578d0'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:16.633923 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-d493ec2e-a320-464a-8348-9c5e871b7381 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16c8b44c5fe446f28ac8dce3f97578d0'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:16.651951 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1068/2117] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:14:16 2022] DELETE /identity/v3/users/16c8b44c5fe446f28ac8dce3f97578d0 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:14:16.692540 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c099b9-8850-4a48-892c-3dc4b35bf32b service cinder] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:16.693953 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-b6c099b9-8850-4a48-892c-3dc4b35bf32b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:16.705303 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c099b9-8850-4a48-892c-3dc4b35bf32b service cinder] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:16.755445 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a9b10f-3105-4c64-9d8b-6e3b3a14d964 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:16.756057 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a9b10f-3105-4c64-9d8b-6e3b3a14d964 service cinder] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:16.756338 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a9b10f-3105-4c64-9d8b-6e3b3a14d964 service cinder] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:16.756609 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a9b10f-3105-4c64-9d8b-6e3b3a14d964 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:16.758637 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-06a9b10f-3105-4c64-9d8b-6e3b3a14d964 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:16.794186 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:16.818332 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a9b10f-3105-4c64-9d8b-6e3b3a14d964 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:16.819860 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a9b10f-3105-4c64-9d8b-6e3b3a14d964 service cinder] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:16.829392 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1050/2118] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:14:16 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:14:17.067583 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d493ec2e-a320-464a-8348-9c5e871b7381 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:17.069205 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-d493ec2e-a320-464a-8348-9c5e871b7381 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:17.165787 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d493ec2e-a320-464a-8348-9c5e871b7381 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:17.170676 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5fe4a2-a75e-495f-9eb9-d0bb8016231a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:17.171420 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5fe4a2-a75e-495f-9eb9-d0bb8016231a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:17.171766 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5fe4a2-a75e-495f-9eb9-d0bb8016231a admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:17.172048 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5fe4a2-a75e-495f-9eb9-d0bb8016231a admin admin] PATH_INFO: `/v3/projects/8450f5fcda4b43bb9f87ad95d0875218` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:17.173179 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5fe4a2-a75e-495f-9eb9-d0bb8016231a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8450f5fcda4b43bb9f87ad95d0875218)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:17.175111 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5fe4a2-a75e-495f-9eb9-d0bb8016231a admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:17.274930 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-2d5fe4a2-a75e-495f-9eb9-d0bb8016231a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8450f5fcda4b43bb9f87ad95d0875218', 'request_id': 'req-2d5fe4a2-a75e-495f-9eb9-d0bb8016231a'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:17.281282 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-2d5fe4a2-a75e-495f-9eb9-d0bb8016231a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8450f5fcda4b43bb9f87ad95d0875218', 'request_id': 'req-2d5fe4a2-a75e-495f-9eb9-d0bb8016231a'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:17.362349 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-2d5fe4a2-a75e-495f-9eb9-d0bb8016231a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8450f5fcda4b43bb9f87ad95d0875218', 'request_id': 'req-2d5fe4a2-a75e-495f-9eb9-d0bb8016231a'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:17.370736 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:17.375012 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-2d5fe4a2-a75e-495f-9eb9-d0bb8016231a admin admin] The token cache is being invalidate because project 8450f5fcda4b43bb9f87ad95d0875218 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:14:17.375305 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-2d5fe4a2-a75e-495f-9eb9-d0bb8016231a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:17.375965 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-2d5fe4a2-a75e-495f-9eb9-d0bb8016231a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:17.377501 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1069/2119] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:14:17 2022] DELETE /identity/v3/projects/8450f5fcda4b43bb9f87ad95d0875218 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:14:17.382276 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c619d7a9-746d-4d59-a149-0854a705d6af None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:17.383340 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c619d7a9-746d-4d59-a149-0854a705d6af None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:17.383679 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c619d7a9-746d-4d59-a149-0854a705d6af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:17.403530 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-c619d7a9-746d-4d59-a149-0854a705d6af None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:14:17.404744 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-c619d7a9-746d-4d59-a149-0854a705d6af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:17.471159 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1051/2120] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:14:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 17:14:17.480718 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37213703-8855-4079-b704-979eb2f515d9 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:17.480718 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37213703-8855-4079-b704-979eb2f515d9 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:17.480718 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37213703-8855-4079-b704-979eb2f515d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:17.512213 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-37213703-8855-4079-b704-979eb2f515d9 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:14:17.513547 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-37213703-8855-4079-b704-979eb2f515d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:17.525530 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1070/2121] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:14:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:14:17.526396 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c619d7a9-746d-4d59-a149-0854a705d6af None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:17.546234 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c619d7a9-746d-4d59-a149-0854a705d6af None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:17.552050 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cdc141-9b3c-41ca-a96f-5c93329638ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:17.552869 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cdc141-9b3c-41ca-a96f-5c93329638ea admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:17.553160 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cdc141-9b3c-41ca-a96f-5c93329638ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:17.553465 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cdc141-9b3c-41ca-a96f-5c93329638ea admin admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:17.554120 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cdc141-9b3c-41ca-a96f-5c93329638ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:14:17.555126 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cdc141-9b3c-41ca-a96f-5c93329638ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:17.557063 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cdc141-9b3c-41ca-a96f-5c93329638ea admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:17.575216 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1052/2122] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:14:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 17:14:17.576181 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37213703-8855-4079-b704-979eb2f515d9 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:17.585489 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37213703-8855-4079-b704-979eb2f515d9 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:17.590694 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a9db50-3a76-47fd-9b07-3b6edc80295e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:17.591446 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9db50-3a76-47fd-9b07-3b6edc80295e admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:17.591810 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9db50-3a76-47fd-9b07-3b6edc80295e admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:17.592268 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9db50-3a76-47fd-9b07-3b6edc80295e admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:17.593211 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a9db50-3a76-47fd-9b07-3b6edc80295e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:17.597013 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a9db50-3a76-47fd-9b07-3b6edc80295e admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:17.699017 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1071/2123] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:14:17 2022] POST /identity/v3/projects => generated 381 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:14:17.702701 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cdc141-9b3c-41ca-a96f-5c93329638ea admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:17.712932 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cdc141-9b3c-41ca-a96f-5c93329638ea admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:17.771134 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdeaf8e-871b-429d-b9b3-1b7939cef1c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:17.771751 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdeaf8e-871b-429d-b9b3-1b7939cef1c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:17.772126 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdeaf8e-871b-429d-b9b3-1b7939cef1c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:17.772394 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdeaf8e-871b-429d-b9b3-1b7939cef1c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:17.773030 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdeaf8e-871b-429d-b9b3-1b7939cef1c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:17.775470 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdeaf8e-871b-429d-b9b3-1b7939cef1c5 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:17.859488 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1053/2124] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:14:17 2022] POST /identity/v3/users => generated 291 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:14:17.863915 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a9db50-3a76-47fd-9b07-3b6edc80295e admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:17.874956 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a9db50-3a76-47fd-9b07-3b6edc80295e admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:17.879494 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428e671f-c827-4dd3-9742-0af4c0861587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:17.880088 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428e671f-c827-4dd3-9742-0af4c0861587 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:17.880374 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428e671f-c827-4dd3-9742-0af4c0861587 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:17.880648 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428e671f-c827-4dd3-9742-0af4c0861587 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:17.881280 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428e671f-c827-4dd3-9742-0af4c0861587 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:17.883129 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428e671f-c827-4dd3-9742-0af4c0861587 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:17.892901 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1072/2125] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:14:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:14:17.896878 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdeaf8e-871b-429d-b9b3-1b7939cef1c5 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:17.906938 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdeaf8e-871b-429d-b9b3-1b7939cef1c5 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:17.911796 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbab941f-9da5-4d57-80c8-60bd7c39f688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:17.912396 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab941f-9da5-4d57-80c8-60bd7c39f688 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:17.912678 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab941f-9da5-4d57-80c8-60bd7c39f688 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:17.912953 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab941f-9da5-4d57-80c8-60bd7c39f688 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:17.913585 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbab941f-9da5-4d57-80c8-60bd7c39f688 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:17.916406 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbab941f-9da5-4d57-80c8-60bd7c39f688 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:17.926876 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1054/2126] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:14:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:14:17.931662 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428e671f-c827-4dd3-9742-0af4c0861587 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:17.941329 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428e671f-c827-4dd3-9742-0af4c0861587 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:17.946068 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3979dabc-adb2-445e-bb95-6848903ccf23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:17.946701 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3979dabc-adb2-445e-bb95-6848903ccf23 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:17.947012 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3979dabc-adb2-445e-bb95-6848903ccf23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:17.947286 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3979dabc-adb2-445e-bb95-6848903ccf23 admin admin] PATH_INFO: `/v3/projects/f3df01cc237b4f9b9b8f1dc3ff522fb5/users/f6faafcb472a4a63b8efe9f5c94a93f9/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:17.967009 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3979dabc-adb2-445e-bb95-6848903ccf23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3df01cc237b4f9b9b8f1dc3ff522fb5, user_id=f6faafcb472a4a63b8efe9f5c94a93f9, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:17.967009 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3979dabc-adb2-445e-bb95-6848903ccf23 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:18.033335 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1073/2127] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:14:17 2022] PUT /identity/v3/projects/f3df01cc237b4f9b9b8f1dc3ff522fb5/users/f6faafcb472a4a63b8efe9f5c94a93f9/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:14:18.046194 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2ad15-eec7-4064-9930-4667dca70d62 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:18.046675 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2ad15-eec7-4064-9930-4667dca70d62 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:18.047247 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2ad15-eec7-4064-9930-4667dca70d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:18.099069 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-8ec2ad15-eec7-4064-9930-4667dca70d62 None None] MFA Rules not processed for user `f6faafcb472a4a63b8efe9f5c94a93f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:14:18.100442 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-8ec2ad15-eec7-4064-9930-4667dca70d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:18.130840 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:18.165002 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1055/2128] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:14:18 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:14:18.177036 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3979dabc-adb2-445e-bb95-6848903ccf23 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:18.178849 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-3979dabc-adb2-445e-bb95-6848903ccf23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:18.241007 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3979dabc-adb2-445e-bb95-6848903ccf23 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:18.241007 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d30a09-e4da-417a-bbb9-33637cf44ddb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:18.241486 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d30a09-e4da-417a-bbb9-33637cf44ddb service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:18.241812 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d30a09-e4da-417a-bbb9-33637cf44ddb service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:18.242144 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d30a09-e4da-417a-bbb9-33637cf44ddb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:18.250684 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d30a09-e4da-417a-bbb9-33637cf44ddb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:18.252162 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d30a09-e4da-417a-bbb9-33637cf44ddb service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:18.261723 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1074/2129] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:14:18 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:14:18.281413 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec2ad15-eec7-4064-9930-4667dca70d62 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:18.349017 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec2ad15-eec7-4064-9930-4667dca70d62 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:18.354590 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6287ba68-9c49-4573-b43f-49d64bfa0fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:18.355248 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6287ba68-9c49-4573-b43f-49d64bfa0fbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:18.355550 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6287ba68-9c49-4573-b43f-49d64bfa0fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:18.355825 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6287ba68-9c49-4573-b43f-49d64bfa0fbb admin admin] PATH_INFO: `/v3/users/f6faafcb472a4a63b8efe9f5c94a93f9` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:18.356945 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6287ba68-9c49-4573-b43f-49d64bfa0fbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6faafcb472a4a63b8efe9f5c94a93f9)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:18.358879 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6287ba68-9c49-4573-b43f-49d64bfa0fbb admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:18.578532 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-6287ba68-9c49-4573-b43f-49d64bfa0fbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6faafcb472a4a63b8efe9f5c94a93f9'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:18.591147 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-6287ba68-9c49-4573-b43f-49d64bfa0fbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6faafcb472a4a63b8efe9f5c94a93f9'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:18.591850 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-6287ba68-9c49-4573-b43f-49d64bfa0fbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6faafcb472a4a63b8efe9f5c94a93f9'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:18.673399 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-6287ba68-9c49-4573-b43f-49d64bfa0fbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6faafcb472a4a63b8efe9f5c94a93f9'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:18.674180 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-6287ba68-9c49-4573-b43f-49d64bfa0fbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6faafcb472a4a63b8efe9f5c94a93f9'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:18.688719 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1056/2130] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:14:18 2022] DELETE /identity/v3/users/f6faafcb472a4a63b8efe9f5c94a93f9 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:14:18.693179 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb1940-f6b1-420c-87ad-e4dc82967e46 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:18.693487 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb1940-f6b1-420c-87ad-e4dc82967e46 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:18.693767 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb1940-f6b1-420c-87ad-e4dc82967e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:18.714170 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-5dcb1940-f6b1-420c-87ad-e4dc82967e46 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:14:18.715424 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-5dcb1940-f6b1-420c-87ad-e4dc82967e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:18.774562 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1075/2131] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:14:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:14:18.784275 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6287ba68-9c49-4573-b43f-49d64bfa0fbb admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:18.785765 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-6287ba68-9c49-4573-b43f-49d64bfa0fbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:18.844290 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6287ba68-9c49-4573-b43f-49d64bfa0fbb admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:18.847988 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0bfc16-574d-4007-aebb-445cb3fcb9f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:18.848591 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0bfc16-574d-4007-aebb-445cb3fcb9f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:18.848870 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0bfc16-574d-4007-aebb-445cb3fcb9f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:18.849140 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0bfc16-574d-4007-aebb-445cb3fcb9f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:18.855200 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:18.858708 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0bfc16-574d-4007-aebb-445cb3fcb9f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:18.860173 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0bfc16-574d-4007-aebb-445cb3fcb9f3 service neutron] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:18.870727 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1057/2132] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:14:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:14:19.294256 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcb1940-f6b1-420c-87ad-e4dc82967e46 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:19.295744 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-5dcb1940-f6b1-420c-87ad-e4dc82967e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:19.315345 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcb1940-f6b1-420c-87ad-e4dc82967e46 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:19.321292 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaa85fe-0979-4071-911b-0de716b9a396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:19.321624 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa85fe-0979-4071-911b-0de716b9a396 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:19.322024 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa85fe-0979-4071-911b-0de716b9a396 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:19.322304 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa85fe-0979-4071-911b-0de716b9a396 admin admin] PATH_INFO: `/v3/projects/f3df01cc237b4f9b9b8f1dc3ff522fb5` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:19.323591 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaa85fe-0979-4071-911b-0de716b9a396 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3df01cc237b4f9b9b8f1dc3ff522fb5)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:19.325490 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaa85fe-0979-4071-911b-0de716b9a396 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:19.431761 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:19.448633 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-deaa85fe-0979-4071-911b-0de716b9a396 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3df01cc237b4f9b9b8f1dc3ff522fb5', 'request_id': 'req-deaa85fe-0979-4071-911b-0de716b9a396'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:19.455228 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-deaa85fe-0979-4071-911b-0de716b9a396 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3df01cc237b4f9b9b8f1dc3ff522fb5', 'request_id': 'req-deaa85fe-0979-4071-911b-0de716b9a396'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:19.510860 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-deaa85fe-0979-4071-911b-0de716b9a396 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3df01cc237b4f9b9b8f1dc3ff522fb5', 'request_id': 'req-deaa85fe-0979-4071-911b-0de716b9a396'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:19.523354 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-deaa85fe-0979-4071-911b-0de716b9a396 admin admin] The token cache is being invalidate because project f3df01cc237b4f9b9b8f1dc3ff522fb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:14:19.523666 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-deaa85fe-0979-4071-911b-0de716b9a396 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:19.524410 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-deaa85fe-0979-4071-911b-0de716b9a396 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:19.525998 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1076/2133] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:14:19 2022] DELETE /identity/v3/projects/f3df01cc237b4f9b9b8f1dc3ff522fb5 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:14:19.530977 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7818f4b7-dc27-45c5-bd33-4dc703becb63 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:19.531297 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7818f4b7-dc27-45c5-bd33-4dc703becb63 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:19.531581 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7818f4b7-dc27-45c5-bd33-4dc703becb63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:19.552100 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-7818f4b7-dc27-45c5-bd33-4dc703becb63 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:14:19.553222 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-7818f4b7-dc27-45c5-bd33-4dc703becb63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:19.631037 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1058/2134] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:14:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 17:14:19.639704 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038160a0-9f52-4e35-8462-b21b780655dc None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:19.639992 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038160a0-9f52-4e35-8462-b21b780655dc None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:19.640268 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038160a0-9f52-4e35-8462-b21b780655dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:19.660467 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-038160a0-9f52-4e35-8462-b21b780655dc None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:14:19.661664 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-038160a0-9f52-4e35-8462-b21b780655dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:19.674043 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1077/2135] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:14:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:14:19.675552 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7818f4b7-dc27-45c5-bd33-4dc703becb63 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:19.691253 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7818f4b7-dc27-45c5-bd33-4dc703becb63 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:19.697294 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31df778-0c01-4b29-ac94-268b62bd689d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:19.703979 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31df778-0c01-4b29-ac94-268b62bd689d admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:19.704356 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31df778-0c01-4b29-ac94-268b62bd689d admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:19.704708 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31df778-0c01-4b29-ac94-268b62bd689d admin admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:19.705514 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31df778-0c01-4b29-ac94-268b62bd689d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:14:19.705894 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31df778-0c01-4b29-ac94-268b62bd689d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:19.707876 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31df778-0c01-4b29-ac94-268b62bd689d admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:19.725437 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1059/2136] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:14:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 17:14:19.728484 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038160a0-9f52-4e35-8462-b21b780655dc None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:19.740213 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038160a0-9f52-4e35-8462-b21b780655dc None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:19.745722 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7709539-2a75-4ffa-93a8-082645c37c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:19.757007 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7709539-2a75-4ffa-93a8-082645c37c08 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:19.757409 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7709539-2a75-4ffa-93a8-082645c37c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:19.758375 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7709539-2a75-4ffa-93a8-082645c37c08 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:19.759413 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7709539-2a75-4ffa-93a8-082645c37c08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:19.761726 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7709539-2a75-4ffa-93a8-082645c37c08 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:19.857219 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1078/2137] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:14:19 2022] POST /identity/v3/projects => generated 371 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:14:19.861905 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31df778-0c01-4b29-ac94-268b62bd689d admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:19.872510 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31df778-0c01-4b29-ac94-268b62bd689d admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:19.931331 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437589cb-484e-4748-9516-25bde7dc6c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:19.931971 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437589cb-484e-4748-9516-25bde7dc6c4a admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:19.932276 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437589cb-484e-4748-9516-25bde7dc6c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:19.932552 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437589cb-484e-4748-9516-25bde7dc6c4a admin admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:19.933166 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437589cb-484e-4748-9516-25bde7dc6c4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:19.935169 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437589cb-484e-4748-9516-25bde7dc6c4a admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:19.964056 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7709539-2a75-4ffa-93a8-082645c37c08 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:19.965761 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-b7709539-2a75-4ffa-93a8-082645c37c08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:19.978943 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7709539-2a75-4ffa-93a8-082645c37c08 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:19.985442 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9494df1e-b823-43bc-aaeb-4e6fb4f173af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:19.991734 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9494df1e-b823-43bc-aaeb-4e6fb4f173af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:19.991734 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9494df1e-b823-43bc-aaeb-4e6fb4f173af admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:19.991734 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9494df1e-b823-43bc-aaeb-4e6fb4f173af admin admin] PATH_INFO: `/v3/users/f515adcb16b64ca88d930abc1840523c` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:19.991734 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9494df1e-b823-43bc-aaeb-4e6fb4f173af admin admin] RBAC: Authorizing `identity:delete_user(user_id=f515adcb16b64ca88d930abc1840523c)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:19.991734 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9494df1e-b823-43bc-aaeb-4e6fb4f173af admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:20.027590 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1060/2138] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:14:19 2022] POST /identity/v3/users => generated 286 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:14:20.028710 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437589cb-484e-4748-9516-25bde7dc6c4a admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:20.038791 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437589cb-484e-4748-9516-25bde7dc6c4a admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:20.044108 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04edd5fd-a3e4-4810-8435-763df2bb1358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:20.044694 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04edd5fd-a3e4-4810-8435-763df2bb1358 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:20.044984 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04edd5fd-a3e4-4810-8435-763df2bb1358 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:20.045259 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04edd5fd-a3e4-4810-8435-763df2bb1358 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:20.045891 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04edd5fd-a3e4-4810-8435-763df2bb1358 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:20.047635 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04edd5fd-a3e4-4810-8435-763df2bb1358 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:20.058453 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1061/2139] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:14:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 17:14:20.062275 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04edd5fd-a3e4-4810-8435-763df2bb1358 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:20.073113 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04edd5fd-a3e4-4810-8435-763df2bb1358 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:20.078120 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82802cdd-230c-45f2-8397-2f0d96764883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:20.078725 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82802cdd-230c-45f2-8397-2f0d96764883 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:20.079042 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82802cdd-230c-45f2-8397-2f0d96764883 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:20.079441 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82802cdd-230c-45f2-8397-2f0d96764883 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:20.080114 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82802cdd-230c-45f2-8397-2f0d96764883 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:20.081707 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82802cdd-230c-45f2-8397-2f0d96764883 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:20.097308 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1062/2140] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:14:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:14:20.098023 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82802cdd-230c-45f2-8397-2f0d96764883 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:20.109838 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82802cdd-230c-45f2-8397-2f0d96764883 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:20.114340 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a73603-79b6-4952-92b8-67e420021fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:20.115057 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a73603-79b6-4952-92b8-67e420021fc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:20.115384 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a73603-79b6-4952-92b8-67e420021fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:20.115606 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a73603-79b6-4952-92b8-67e420021fc5 admin admin] PATH_INFO: `/v3/projects/9e161860d18f4d608a9ce4177cce7b6f/users/497f5cef83ff4fe69e00e3ccf703088d/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:20.130368 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a73603-79b6-4952-92b8-67e420021fc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e161860d18f4d608a9ce4177cce7b6f, user_id=497f5cef83ff4fe69e00e3ccf703088d, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:20.133782 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a73603-79b6-4952-92b8-67e420021fc5 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:20.198648 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-9494df1e-b823-43bc-aaeb-4e6fb4f173af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f515adcb16b64ca88d930abc1840523c'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:20.232476 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1063/2141] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:14:20 2022] PUT /identity/v3/projects/9e161860d18f4d608a9ce4177cce7b6f/users/497f5cef83ff4fe69e00e3ccf703088d/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:14:20.237877 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db43987c-182c-415e-945e-d566f0c81851 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:20.238173 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db43987c-182c-415e-945e-d566f0c81851 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:20.238453 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db43987c-182c-415e-945e-d566f0c81851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:20.276288 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-db43987c-182c-415e-945e-d566f0c81851 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:14:20.277541 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-db43987c-182c-415e-945e-d566f0c81851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:20.299472 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:20.311347 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-9494df1e-b823-43bc-aaeb-4e6fb4f173af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f515adcb16b64ca88d930abc1840523c'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:20.312530 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-9494df1e-b823-43bc-aaeb-4e6fb4f173af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f515adcb16b64ca88d930abc1840523c'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:20.313200 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-9494df1e-b823-43bc-aaeb-4e6fb4f173af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f515adcb16b64ca88d930abc1840523c'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:20.324566 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-9494df1e-b823-43bc-aaeb-4e6fb4f173af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f515adcb16b64ca88d930abc1840523c'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:20.349482 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1064/2142] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:14:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 17:14:20.350137 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1079/2143] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:14:19 2022] DELETE /identity/v3/users/f515adcb16b64ca88d930abc1840523c => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:14:20.355599 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c72491-f6ac-4327-bd03-9c278daf592d None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:20.356009 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c72491-f6ac-4327-bd03-9c278daf592d None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:20.356360 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c72491-f6ac-4327-bd03-9c278daf592d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:20.360553 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9494df1e-b823-43bc-aaeb-4e6fb4f173af admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:20.364086 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-9494df1e-b823-43bc-aaeb-4e6fb4f173af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:20.377584 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-b3c72491-f6ac-4327-bd03-9c278daf592d None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:14:20.378813 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-b3c72491-f6ac-4327-bd03-9c278daf592d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:20.441633 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9494df1e-b823-43bc-aaeb-4e6fb4f173af admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:20.446125 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c0ade7-38d2-4e42-a76c-a95c69b9a6a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:20.446781 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c0ade7-38d2-4e42-a76c-a95c69b9a6a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:20.447091 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c0ade7-38d2-4e42-a76c-a95c69b9a6a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:20.447363 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c0ade7-38d2-4e42-a76c-a95c69b9a6a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:20.449347 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1065/2144] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:14:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 17:14:20.456680 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c0ade7-38d2-4e42-a76c-a95c69b9a6a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:20.458107 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c0ade7-38d2-4e42-a76c-a95c69b9a6a7 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:20.459906 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c72491-f6ac-4327-bd03-9c278daf592d None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:20.469162 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1080/2145] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:14:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:14:20.470469 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c72491-f6ac-4327-bd03-9c278daf592d None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:20.474103 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0abddb9-d80b-4767-af33-d294f0252a02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:20.474707 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0abddb9-d80b-4767-af33-d294f0252a02 service neutron] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:20.475031 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0abddb9-d80b-4767-af33-d294f0252a02 service neutron] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:20.475316 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0abddb9-d80b-4767-af33-d294f0252a02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:20.485128 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0abddb9-d80b-4767-af33-d294f0252a02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:20.486594 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0abddb9-d80b-4767-af33-d294f0252a02 service neutron] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:20.497061 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1066/2146] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:14:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:14:20.979485 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c0ade7-38d2-4e42-a76c-a95c69b9a6a7 service neutron] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:20.982538 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-d2c0ade7-38d2-4e42-a76c-a95c69b9a6a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:21.001690 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c0ade7-38d2-4e42-a76c-a95c69b9a6a7 service neutron] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:21.010979 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a34d6b-b2c2-41b1-8c7f-505c54d639f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:21.011721 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a34d6b-b2c2-41b1-8c7f-505c54d639f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:21.012003 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a34d6b-b2c2-41b1-8c7f-505c54d639f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:21.012325 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a34d6b-b2c2-41b1-8c7f-505c54d639f4 admin admin] PATH_INFO: `/v3/projects/3d406a416a48402588b0c57278139ad4` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:21.014107 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a34d6b-b2c2-41b1-8c7f-505c54d639f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d406a416a48402588b0c57278139ad4)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:21.016098 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a34d6b-b2c2-41b1-8c7f-505c54d639f4 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:21.019871 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:DELETE=15,keystone:SELECT=82,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:21.150609 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-b4a34d6b-b2c2-41b1-8c7f-505c54d639f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d406a416a48402588b0c57278139ad4', 'request_id': 'req-b4a34d6b-b2c2-41b1-8c7f-505c54d639f4'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:21.157825 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-b4a34d6b-b2c2-41b1-8c7f-505c54d639f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d406a416a48402588b0c57278139ad4', 'request_id': 'req-b4a34d6b-b2c2-41b1-8c7f-505c54d639f4'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:21.262185 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-b4a34d6b-b2c2-41b1-8c7f-505c54d639f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d406a416a48402588b0c57278139ad4', 'request_id': 'req-b4a34d6b-b2c2-41b1-8c7f-505c54d639f4'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:21.283409 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-b4a34d6b-b2c2-41b1-8c7f-505c54d639f4 admin admin] The token cache is being invalidate because project 3d406a416a48402588b0c57278139ad4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:14:21.283709 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-b4a34d6b-b2c2-41b1-8c7f-505c54d639f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:21.286829 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-b4a34d6b-b2c2-41b1-8c7f-505c54d639f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:21.298773 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1081/2147] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:14:20 2022] DELETE /identity/v3/projects/3d406a416a48402588b0c57278139ad4 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:14:21.881260 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac04a855-c23d-433a-ac4b-a8452e2029b6 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:21.881900 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac04a855-c23d-433a-ac4b-a8452e2029b6 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:21.882769 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac04a855-c23d-433a-ac4b-a8452e2029b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:21.913264 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-ac04a855-c23d-433a-ac4b-a8452e2029b6 None None] MFA Rules not processed for user `497f5cef83ff4fe69e00e3ccf703088d`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:14:21.914614 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-ac04a855-c23d-433a-ac4b-a8452e2029b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:21.995886 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1067/2148] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:14:21 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 17:14:22.008219 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a34d6b-b2c2-41b1-8c7f-505c54d639f4 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:22.010765 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-b4a34d6b-b2c2-41b1-8c7f-505c54d639f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:22.072421 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a34d6b-b2c2-41b1-8c7f-505c54d639f4 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:22.076052 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea9c4e0-a6fc-4a65-ab4c-ebe9cbfa61ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:22.076665 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea9c4e0-a6fc-4a65-ab4c-ebe9cbfa61ef service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:22.076945 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea9c4e0-a6fc-4a65-ab4c-ebe9cbfa61ef service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:22.077219 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea9c4e0-a6fc-4a65-ab4c-ebe9cbfa61ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:22.085706 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea9c4e0-a6fc-4a65-ab4c-ebe9cbfa61ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:22.087180 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea9c4e0-a6fc-4a65-ab4c-ebe9cbfa61ef service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:22.096214 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1082/2149] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:14:22 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:14:22.189854 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac04a855-c23d-433a-ac4b-a8452e2029b6 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:22.191350 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-ac04a855-c23d-433a-ac4b-a8452e2029b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:22.239539 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:22.259005 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac04a855-c23d-433a-ac4b-a8452e2029b6 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:22.263599 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d304e09-5505-43f9-96ae-81be958e993c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:22.264232 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d304e09-5505-43f9-96ae-81be958e993c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:22.264520 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d304e09-5505-43f9-96ae-81be958e993c admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:22.264797 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d304e09-5505-43f9-96ae-81be958e993c admin admin] PATH_INFO: `/v3/users/1542f734d249496e90d8278a95d285a3` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:22.265955 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d304e09-5505-43f9-96ae-81be958e993c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1542f734d249496e90d8278a95d285a3)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:22.267962 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d304e09-5505-43f9-96ae-81be958e993c admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:22.518243 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-6d304e09-5505-43f9-96ae-81be958e993c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1542f734d249496e90d8278a95d285a3'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:22.530155 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-6d304e09-5505-43f9-96ae-81be958e993c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1542f734d249496e90d8278a95d285a3'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:22.530828 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-6d304e09-5505-43f9-96ae-81be958e993c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1542f734d249496e90d8278a95d285a3'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:22.627654 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-6d304e09-5505-43f9-96ae-81be958e993c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1542f734d249496e90d8278a95d285a3'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:22.628471 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-6d304e09-5505-43f9-96ae-81be958e993c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1542f734d249496e90d8278a95d285a3'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:22.642300 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1068/2150] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:14:22 2022] DELETE /identity/v3/users/1542f734d249496e90d8278a95d285a3 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:14:22.647060 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4154a18b-aab4-477f-ac0d-731ed6ecc576 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:22.647410 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4154a18b-aab4-477f-ac0d-731ed6ecc576 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:22.648254 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4154a18b-aab4-477f-ac0d-731ed6ecc576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:22.668199 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-4154a18b-aab4-477f-ac0d-731ed6ecc576 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:14:22.669392 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-4154a18b-aab4-477f-ac0d-731ed6ecc576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:22.734746 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1083/2151] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:14:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:14:22.760418 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d304e09-5505-43f9-96ae-81be958e993c admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:22.763092 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-6d304e09-5505-43f9-96ae-81be958e993c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:22.826521 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d304e09-5505-43f9-96ae-81be958e993c admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:22.830072 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0046c768-b34b-4f2a-861b-5fa8c0aca7b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:22.830675 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0046c768-b34b-4f2a-861b-5fa8c0aca7b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:22.830986 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0046c768-b34b-4f2a-861b-5fa8c0aca7b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:22.831592 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0046c768-b34b-4f2a-861b-5fa8c0aca7b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:22.841417 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0046c768-b34b-4f2a-861b-5fa8c0aca7b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:22.842856 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0046c768-b34b-4f2a-861b-5fa8c0aca7b4 service neutron] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:22.853579 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1069/2152] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:14:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:14:23.261333 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4154a18b-aab4-477f-ac0d-731ed6ecc576 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:23.263292 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-4154a18b-aab4-477f-ac0d-731ed6ecc576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:23.278559 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4154a18b-aab4-477f-ac0d-731ed6ecc576 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:23.284197 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656f3c8e-23e0-4570-914a-eadcdc319833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:23.284835 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f3c8e-23e0-4570-914a-eadcdc319833 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:23.285116 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f3c8e-23e0-4570-914a-eadcdc319833 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:23.285415 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f3c8e-23e0-4570-914a-eadcdc319833 admin admin] PATH_INFO: `/v3/projects/ee1edd3e6d4e42a18f345c846a484b73` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:23.287410 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656f3c8e-23e0-4570-914a-eadcdc319833 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee1edd3e6d4e42a18f345c846a484b73)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:23.291077 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656f3c8e-23e0-4570-914a-eadcdc319833 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:23.420235 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:23.433448 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-656f3c8e-23e0-4570-914a-eadcdc319833 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee1edd3e6d4e42a18f345c846a484b73', 'request_id': 'req-656f3c8e-23e0-4570-914a-eadcdc319833'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:23.439834 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-656f3c8e-23e0-4570-914a-eadcdc319833 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee1edd3e6d4e42a18f345c846a484b73', 'request_id': 'req-656f3c8e-23e0-4570-914a-eadcdc319833'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:23.547428 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-656f3c8e-23e0-4570-914a-eadcdc319833 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee1edd3e6d4e42a18f345c846a484b73', 'request_id': 'req-656f3c8e-23e0-4570-914a-eadcdc319833'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:23.560066 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-656f3c8e-23e0-4570-914a-eadcdc319833 admin admin] The token cache is being invalidate because project ee1edd3e6d4e42a18f345c846a484b73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:14:23.560364 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-656f3c8e-23e0-4570-914a-eadcdc319833 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:23.561317 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-656f3c8e-23e0-4570-914a-eadcdc319833 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:23.563501 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1084/2153] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:14:23 2022] DELETE /identity/v3/projects/ee1edd3e6d4e42a18f345c846a484b73 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:14:23.567297 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0046c768-b34b-4f2a-861b-5fa8c0aca7b4 service neutron] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:23.577712 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0046c768-b34b-4f2a-861b-5fa8c0aca7b4 service neutron] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:23.660372 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf533033-fa90-4fd4-9ced-bc773a5182b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:23.661409 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf533033-fa90-4fd4-9ced-bc773a5182b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:23.661748 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf533033-fa90-4fd4-9ced-bc773a5182b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:23.662026 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf533033-fa90-4fd4-9ced-bc773a5182b4 admin admin] PATH_INFO: `/v3/users/b186d23bf56f4aacacd9f10d17391079` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:23.664175 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf533033-fa90-4fd4-9ced-bc773a5182b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b186d23bf56f4aacacd9f10d17391079)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:23.667036 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf533033-fa90-4fd4-9ced-bc773a5182b4 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:23.673670 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:23.876417 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-bf533033-fa90-4fd4-9ced-bc773a5182b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b186d23bf56f4aacacd9f10d17391079'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:23.896326 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-bf533033-fa90-4fd4-9ced-bc773a5182b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b186d23bf56f4aacacd9f10d17391079'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:23.897440 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-bf533033-fa90-4fd4-9ced-bc773a5182b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b186d23bf56f4aacacd9f10d17391079'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:23.981310 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-bf533033-fa90-4fd4-9ced-bc773a5182b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b186d23bf56f4aacacd9f10d17391079'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:23.982067 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-bf533033-fa90-4fd4-9ced-bc773a5182b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b186d23bf56f4aacacd9f10d17391079'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:23.996544 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1070/2154] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:14:23 2022] DELETE /identity/v3/users/b186d23bf56f4aacacd9f10d17391079 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:14:24.645778 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656f3c8e-23e0-4570-914a-eadcdc319833 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:24.647322 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-656f3c8e-23e0-4570-914a-eadcdc319833 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:24.662045 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656f3c8e-23e0-4570-914a-eadcdc319833 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:24.720266 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f052cf5e-69cd-4e99-866d-46e5f3e0c9e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:24.721710 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f052cf5e-69cd-4e99-866d-46e5f3e0c9e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:24.722011 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f052cf5e-69cd-4e99-866d-46e5f3e0c9e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:24.722629 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f052cf5e-69cd-4e99-866d-46e5f3e0c9e4 admin admin] PATH_INFO: `/v3/projects/41bb531a07744267a28589bae4907f4e` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:24.723800 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f052cf5e-69cd-4e99-866d-46e5f3e0c9e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41bb531a07744267a28589bae4907f4e)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:24.726329 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f052cf5e-69cd-4e99-866d-46e5f3e0c9e4 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:24.862264 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-f052cf5e-69cd-4e99-866d-46e5f3e0c9e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41bb531a07744267a28589bae4907f4e', 'request_id': 'req-f052cf5e-69cd-4e99-866d-46e5f3e0c9e4'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:24.868597 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-f052cf5e-69cd-4e99-866d-46e5f3e0c9e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41bb531a07744267a28589bae4907f4e', 'request_id': 'req-f052cf5e-69cd-4e99-866d-46e5f3e0c9e4'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:24.962530 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d1915-98b0-44dd-87b0-f9a781de49a8 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:24.963088 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d1915-98b0-44dd-87b0-f9a781de49a8 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:24.963205 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d1915-98b0-44dd-87b0-f9a781de49a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:24.979479 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-f052cf5e-69cd-4e99-866d-46e5f3e0c9e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41bb531a07744267a28589bae4907f4e', 'request_id': 'req-f052cf5e-69cd-4e99-866d-46e5f3e0c9e4'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:24.988135 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-a51d1915-98b0-44dd-87b0-f9a781de49a8 None None] MFA Rules not processed for user `497f5cef83ff4fe69e00e3ccf703088d`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:14:24.989460 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-a51d1915-98b0-44dd-87b0-f9a781de49a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:24.995902 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-f052cf5e-69cd-4e99-866d-46e5f3e0c9e4 admin admin] The token cache is being invalidate because project 41bb531a07744267a28589bae4907f4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:14:24.996817 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-f052cf5e-69cd-4e99-866d-46e5f3e0c9e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:24.997762 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-f052cf5e-69cd-4e99-866d-46e5f3e0c9e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:25.000461 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1085/2155] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:14:24 2022] DELETE /identity/v3/projects/41bb531a07744267a28589bae4907f4e => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:14:25.006348 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f052cf5e-69cd-4e99-866d-46e5f3e0c9e4 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:25.008527 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-f052cf5e-69cd-4e99-866d-46e5f3e0c9e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:25.071113 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:25.091942 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f052cf5e-69cd-4e99-866d-46e5f3e0c9e4 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:25.097485 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5659d36c-71f8-4837-a292-6ea9a741d76d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:25.098125 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5659d36c-71f8-4837-a292-6ea9a741d76d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:25.098409 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5659d36c-71f8-4837-a292-6ea9a741d76d admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:25.098537 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1071/2156] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:14:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 17:14:25.098682 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5659d36c-71f8-4837-a292-6ea9a741d76d admin admin] PATH_INFO: `/v3/users/7ad8931447964e1a9fcdf5b8238910ad` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:25.099862 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5659d36c-71f8-4837-a292-6ea9a741d76d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ad8931447964e1a9fcdf5b8238910ad)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:25.102348 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5659d36c-71f8-4837-a292-6ea9a741d76d admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:25.299478 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-5659d36c-71f8-4837-a292-6ea9a741d76d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ad8931447964e1a9fcdf5b8238910ad'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:25.368044 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-5659d36c-71f8-4837-a292-6ea9a741d76d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ad8931447964e1a9fcdf5b8238910ad'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:25.368743 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-5659d36c-71f8-4837-a292-6ea9a741d76d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ad8931447964e1a9fcdf5b8238910ad'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:25.369382 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-5659d36c-71f8-4837-a292-6ea9a741d76d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ad8931447964e1a9fcdf5b8238910ad'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:25.377025 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-5659d36c-71f8-4837-a292-6ea9a741d76d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ad8931447964e1a9fcdf5b8238910ad'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:25.396183 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1086/2157] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:14:25 2022] DELETE /identity/v3/users/7ad8931447964e1a9fcdf5b8238910ad => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:14:25.892256 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51d1915-98b0-44dd-87b0-f9a781de49a8 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:25.899807 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51d1915-98b0-44dd-87b0-f9a781de49a8 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:25.964523 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa79fba-67bd-492e-bfc9-c7c631ae4e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:25.965233 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa79fba-67bd-492e-bfc9-c7c631ae4e6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:25.965577 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa79fba-67bd-492e-bfc9-c7c631ae4e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:25.965951 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa79fba-67bd-492e-bfc9-c7c631ae4e6a admin admin] PATH_INFO: `/v3/projects/d8d68a86022a41928a421d4cbeb33b31` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:25.967199 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa79fba-67bd-492e-bfc9-c7c631ae4e6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8d68a86022a41928a421d4cbeb33b31)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:25.969181 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa79fba-67bd-492e-bfc9-c7c631ae4e6a admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:26.073186 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:26.086575 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-ffa79fba-67bd-492e-bfc9-c7c631ae4e6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8d68a86022a41928a421d4cbeb33b31', 'request_id': 'req-ffa79fba-67bd-492e-bfc9-c7c631ae4e6a'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:26.155595 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-ffa79fba-67bd-492e-bfc9-c7c631ae4e6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8d68a86022a41928a421d4cbeb33b31', 'request_id': 'req-ffa79fba-67bd-492e-bfc9-c7c631ae4e6a'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:26.162721 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-ffa79fba-67bd-492e-bfc9-c7c631ae4e6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8d68a86022a41928a421d4cbeb33b31', 'request_id': 'req-ffa79fba-67bd-492e-bfc9-c7c631ae4e6a'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:26.176655 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-ffa79fba-67bd-492e-bfc9-c7c631ae4e6a admin admin] The token cache is being invalidate because project d8d68a86022a41928a421d4cbeb33b31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81650) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:14:26.177047 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-ffa79fba-67bd-492e-bfc9-c7c631ae4e6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:26.178282 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-ffa79fba-67bd-492e-bfc9-c7c631ae4e6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:26.180927 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1072/2158] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:14:25 2022] DELETE /identity/v3/projects/d8d68a86022a41928a421d4cbeb33b31 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:14:26.186677 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6088ae50-f77a-4257-9e49-671c5455bf03 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:26.187216 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6088ae50-f77a-4257-9e49-671c5455bf03 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:26.187339 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6088ae50-f77a-4257-9e49-671c5455bf03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:26.208574 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-6088ae50-f77a-4257-9e49-671c5455bf03 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:14:26.209403 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-6088ae50-f77a-4257-9e49-671c5455bf03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:26.289801 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1087/2159] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:14:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 17:14:26.299673 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e19e5-9977-4a37-a95a-f6d10d04123a None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:26.300175 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e19e5-9977-4a37-a95a-f6d10d04123a None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:26.300452 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e19e5-9977-4a37-a95a-f6d10d04123a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:26.321358 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-c77e19e5-9977-4a37-a95a-f6d10d04123a None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:14:26.322577 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-c77e19e5-9977-4a37-a95a-f6d10d04123a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:26.332744 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1073/2160] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:14:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:14:26.337646 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6088ae50-f77a-4257-9e49-671c5455bf03 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:26.350224 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6088ae50-f77a-4257-9e49-671c5455bf03 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:26.360922 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1e58b2-2783-4f52-a799-dea76edfff9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:26.361570 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1e58b2-2783-4f52-a799-dea76edfff9c admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:26.361859 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1e58b2-2783-4f52-a799-dea76edfff9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:26.362130 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1e58b2-2783-4f52-a799-dea76edfff9c admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:26.362824 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1e58b2-2783-4f52-a799-dea76edfff9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:14:26.363252 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1e58b2-2783-4f52-a799-dea76edfff9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:26.364930 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1e58b2-2783-4f52-a799-dea76edfff9c admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:26.384653 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1088/2161] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:14:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 17:14:26.384786 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77e19e5-9977-4a37-a95a-f6d10d04123a None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:26.395263 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77e19e5-9977-4a37-a95a-f6d10d04123a None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:26.401814 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850fe090-b015-486b-8eea-deba945c6851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:26.402431 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850fe090-b015-486b-8eea-deba945c6851 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:26.402709 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850fe090-b015-486b-8eea-deba945c6851 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:26.403080 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850fe090-b015-486b-8eea-deba945c6851 admin admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:26.403716 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850fe090-b015-486b-8eea-deba945c6851 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:26.405623 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850fe090-b015-486b-8eea-deba945c6851 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:26.496583 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1074/2162] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:14:26 2022] POST /identity/v3/projects => generated 373 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:14:26.500489 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1e58b2-2783-4f52-a799-dea76edfff9c admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:26.510815 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1e58b2-2783-4f52-a799-dea76edfff9c admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:26.570989 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:26.574957 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4624ce4-0d28-4706-926d-441fb1fb22af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:26.575756 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4624ce4-0d28-4706-926d-441fb1fb22af admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:26.576116 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4624ce4-0d28-4706-926d-441fb1fb22af admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:26.576416 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4624ce4-0d28-4706-926d-441fb1fb22af admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:26.577092 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4624ce4-0d28-4706-926d-441fb1fb22af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:26.579438 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4624ce4-0d28-4706-926d-441fb1fb22af admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:26.676641 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1089/2163] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:14:26 2022] POST /identity/v3/users => generated 287 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:14:26.681277 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850fe090-b015-486b-8eea-deba945c6851 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:26.693807 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850fe090-b015-486b-8eea-deba945c6851 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:26.701165 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e422f982-5a9b-4868-9c6d-8c7040c1a2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:26.701781 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e422f982-5a9b-4868-9c6d-8c7040c1a2d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:26.702061 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e422f982-5a9b-4868-9c6d-8c7040c1a2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:26.702328 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e422f982-5a9b-4868-9c6d-8c7040c1a2d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:26.703086 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e422f982-5a9b-4868-9c6d-8c7040c1a2d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:26.704990 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e422f982-5a9b-4868-9c6d-8c7040c1a2d9 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:26.716677 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1075/2164] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 17:14:26.720855 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4624ce4-0d28-4706-926d-441fb1fb22af admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:26.731705 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4624ce4-0d28-4706-926d-441fb1fb22af admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:26.736393 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57fd938-8b7f-4c93-9255-bfd11922050b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:26.736996 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57fd938-8b7f-4c93-9255-bfd11922050b admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:26.737277 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57fd938-8b7f-4c93-9255-bfd11922050b admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:26.737551 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57fd938-8b7f-4c93-9255-bfd11922050b admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:26.738191 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57fd938-8b7f-4c93-9255-bfd11922050b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:26.739889 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57fd938-8b7f-4c93-9255-bfd11922050b admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:26.751264 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1090/2165] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:14:26.755362 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e422f982-5a9b-4868-9c6d-8c7040c1a2d9 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:26.775701 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e422f982-5a9b-4868-9c6d-8c7040c1a2d9 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:26.785930 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b240c9ba-c12e-4460-9929-37e8ed76b35c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:26.786622 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b240c9ba-c12e-4460-9929-37e8ed76b35c admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:26.786933 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b240c9ba-c12e-4460-9929-37e8ed76b35c admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:26.787193 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b240c9ba-c12e-4460-9929-37e8ed76b35c admin admin] PATH_INFO: `/v3/projects/fe80bc385724459493d88db98a3f819a/users/b0fd64da73de40fa9b1c2f97a26a67ae/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:26.812650 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b240c9ba-c12e-4460-9929-37e8ed76b35c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe80bc385724459493d88db98a3f819a, user_id=b0fd64da73de40fa9b1c2f97a26a67ae, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:26.816228 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b240c9ba-c12e-4460-9929-37e8ed76b35c admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:26.893647 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1076/2166] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:14:26 2022] PUT /identity/v3/projects/fe80bc385724459493d88db98a3f819a/users/b0fd64da73de40fa9b1c2f97a26a67ae/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:14:26.905947 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed4495-0260-46f0-8553-05250714b737 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:26.906233 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed4495-0260-46f0-8553-05250714b737 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:26.906527 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed4495-0260-46f0-8553-05250714b737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:26.934999 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-5fed4495-0260-46f0-8553-05250714b737 None None] MFA Rules not processed for user `b0fd64da73de40fa9b1c2f97a26a67ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:14:26.936238 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-5fed4495-0260-46f0-8553-05250714b737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:27.004916 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1091/2167] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:14:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:14:27.017502 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b240c9ba-c12e-4460-9929-37e8ed76b35c admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:27.019277 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-b240c9ba-c12e-4460-9929-37e8ed76b35c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:27.080288 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b240c9ba-c12e-4460-9929-37e8ed76b35c admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:27.084039 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681535f4-9715-41da-a419-e6373294e584 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:27.084648 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681535f4-9715-41da-a419-e6373294e584 service cinder] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:27.085607 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681535f4-9715-41da-a419-e6373294e584 service cinder] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:27.085894 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681535f4-9715-41da-a419-e6373294e584 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:27.094037 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681535f4-9715-41da-a419-e6373294e584 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:27.095519 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681535f4-9715-41da-a419-e6373294e584 service cinder] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:27.104188 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1077/2168] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:14:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:14:27.129950 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fed4495-0260-46f0-8553-05250714b737 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:27.204556 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fed4495-0260-46f0-8553-05250714b737 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:27.211297 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4f7743-a7c1-4c76-96c3-4a17ae6e0bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:27.211928 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f7743-a7c1-4c76-96c3-4a17ae6e0bba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:27.212293 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f7743-a7c1-4c76-96c3-4a17ae6e0bba admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:27.212592 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f7743-a7c1-4c76-96c3-4a17ae6e0bba admin admin] PATH_INFO: `/v3/users/b0fd64da73de40fa9b1c2f97a26a67ae` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:27.213772 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4f7743-a7c1-4c76-96c3-4a17ae6e0bba admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0fd64da73de40fa9b1c2f97a26a67ae)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:27.215720 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4f7743-a7c1-4c76-96c3-4a17ae6e0bba admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:27.411940 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-9e4f7743-a7c1-4c76-96c3-4a17ae6e0bba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0fd64da73de40fa9b1c2f97a26a67ae'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:27.523888 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-9e4f7743-a7c1-4c76-96c3-4a17ae6e0bba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0fd64da73de40fa9b1c2f97a26a67ae'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:27.524507 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-9e4f7743-a7c1-4c76-96c3-4a17ae6e0bba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0fd64da73de40fa9b1c2f97a26a67ae'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:27.525143 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-9e4f7743-a7c1-4c76-96c3-4a17ae6e0bba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0fd64da73de40fa9b1c2f97a26a67ae'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:27.532449 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-9e4f7743-a7c1-4c76-96c3-4a17ae6e0bba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0fd64da73de40fa9b1c2f97a26a67ae'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:27.534830 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:27.558616 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1092/2169] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:14:27 2022] DELETE /identity/v3/users/b0fd64da73de40fa9b1c2f97a26a67ae => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:14:27.563186 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba124d-1ef0-422b-bd1f-20d2ff3d6e33 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:27.563474 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba124d-1ef0-422b-bd1f-20d2ff3d6e33 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:27.563753 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba124d-1ef0-422b-bd1f-20d2ff3d6e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:27.582522 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-d4ba124d-1ef0-422b-bd1f-20d2ff3d6e33 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:14:27.583759 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-d4ba124d-1ef0-422b-bd1f-20d2ff3d6e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:27.636307 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:27.646672 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1078/2170] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:14:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:14:27.656440 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4f7743-a7c1-4c76-96c3-4a17ae6e0bba admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:27.657849 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-9e4f7743-a7c1-4c76-96c3-4a17ae6e0bba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:27.717916 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4f7743-a7c1-4c76-96c3-4a17ae6e0bba admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:27.721819 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6ebeab-5298-4130-9fa9-f6491428a230 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:27.722422 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ebeab-5298-4130-9fa9-f6491428a230 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:27.722704 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ebeab-5298-4130-9fa9-f6491428a230 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:27.723130 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ebeab-5298-4130-9fa9-f6491428a230 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:27.733866 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6ebeab-5298-4130-9fa9-f6491428a230 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:27.735355 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6ebeab-5298-4130-9fa9-f6491428a230 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:27.746362 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1093/2171] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:14:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 17:14:28.565404 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba124d-1ef0-422b-bd1f-20d2ff3d6e33 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:28.566869 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-d4ba124d-1ef0-422b-bd1f-20d2ff3d6e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:28.579774 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba124d-1ef0-422b-bd1f-20d2ff3d6e33 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:28.585567 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0771c80-93fd-4040-a8c4-9a1a668914ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:28.585872 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0771c80-93fd-4040-a8c4-9a1a668914ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:28.586230 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0771c80-93fd-4040-a8c4-9a1a668914ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:28.586580 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0771c80-93fd-4040-a8c4-9a1a668914ce admin admin] PATH_INFO: `/v3/projects/fe80bc385724459493d88db98a3f819a` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:28.587788 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0771c80-93fd-4040-a8c4-9a1a668914ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe80bc385724459493d88db98a3f819a)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:28.589732 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0771c80-93fd-4040-a8c4-9a1a668914ce admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:28.712897 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-c0771c80-93fd-4040-a8c4-9a1a668914ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe80bc385724459493d88db98a3f819a', 'request_id': 'req-c0771c80-93fd-4040-a8c4-9a1a668914ce'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:28.790724 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-c0771c80-93fd-4040-a8c4-9a1a668914ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe80bc385724459493d88db98a3f819a', 'request_id': 'req-c0771c80-93fd-4040-a8c4-9a1a668914ce'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:28.796834 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-c0771c80-93fd-4040-a8c4-9a1a668914ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe80bc385724459493d88db98a3f819a', 'request_id': 'req-c0771c80-93fd-4040-a8c4-9a1a668914ce'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:28.809324 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-c0771c80-93fd-4040-a8c4-9a1a668914ce admin admin] The token cache is being invalidate because project fe80bc385724459493d88db98a3f819a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81650) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:14:28.812768 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-c0771c80-93fd-4040-a8c4-9a1a668914ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:28.812768 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-c0771c80-93fd-4040-a8c4-9a1a668914ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:28.812768 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1079/2172] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:14:28 2022] DELETE /identity/v3/projects/fe80bc385724459493d88db98a3f819a => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:14:28.816770 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847dccb0-2675-4f85-b9f4-02fcf7367d0d None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:28.817263 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847dccb0-2675-4f85-b9f4-02fcf7367d0d None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:28.817362 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847dccb0-2675-4f85-b9f4-02fcf7367d0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:28.838512 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-847dccb0-2675-4f85-b9f4-02fcf7367d0d None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:14:28.839759 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-847dccb0-2675-4f85-b9f4-02fcf7367d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:28.906245 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1094/2173] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:14:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:14:28.914245 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac90e1-8551-424b-872b-fd8222665543 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:28.915935 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac90e1-8551-424b-872b-fd8222665543 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:28.915935 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac90e1-8551-424b-872b-fd8222665543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:28.936004 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-1bac90e1-8551-424b-872b-fd8222665543 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:14:28.938842 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-1bac90e1-8551-424b-872b-fd8222665543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:28.946057 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1080/2174] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:14:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:14:28.950375 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847dccb0-2675-4f85-b9f4-02fcf7367d0d None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:28.960101 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847dccb0-2675-4f85-b9f4-02fcf7367d0d None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:28.964640 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4819ef9d-6131-4206-877c-97409b40f8b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:28.967577 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4819ef9d-6131-4206-877c-97409b40f8b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:28.967577 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4819ef9d-6131-4206-877c-97409b40f8b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:28.967577 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4819ef9d-6131-4206-877c-97409b40f8b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:28.967577 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4819ef9d-6131-4206-877c-97409b40f8b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:14:28.967577 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4819ef9d-6131-4206-877c-97409b40f8b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:28.968317 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4819ef9d-6131-4206-877c-97409b40f8b3 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:28.980143 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1095/2175] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:14:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 17:14:28.984475 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bac90e1-8551-424b-872b-fd8222665543 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:28.994500 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bac90e1-8551-424b-872b-fd8222665543 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:28.999136 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b361f-b0e2-45da-98ec-7f9ad1c35cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:29.001981 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b361f-b0e2-45da-98ec-7f9ad1c35cba admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:29.001981 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b361f-b0e2-45da-98ec-7f9ad1c35cba admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:29.001981 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b361f-b0e2-45da-98ec-7f9ad1c35cba admin admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:29.001981 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61b361f-b0e2-45da-98ec-7f9ad1c35cba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:29.002762 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61b361f-b0e2-45da-98ec-7f9ad1c35cba admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:29.097674 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1081/2176] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:14:28 2022] POST /identity/v3/projects => generated 383 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 17:14:29.102311 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4819ef9d-6131-4206-877c-97409b40f8b3 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:29.117350 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4819ef9d-6131-4206-877c-97409b40f8b3 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:29.213675 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae6f109-aa94-4071-b0b7-3b3637cd868a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:29.214739 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae6f109-aa94-4071-b0b7-3b3637cd868a admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:29.214868 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae6f109-aa94-4071-b0b7-3b3637cd868a admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:29.215093 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae6f109-aa94-4071-b0b7-3b3637cd868a admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:29.215741 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae6f109-aa94-4071-b0b7-3b3637cd868a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:29.217991 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae6f109-aa94-4071-b0b7-3b3637cd868a admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:29.316723 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1096/2177] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:14:29 2022] POST /identity/v3/users => generated 292 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:14:29.323535 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b361f-b0e2-45da-98ec-7f9ad1c35cba admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:29.333689 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b361f-b0e2-45da-98ec-7f9ad1c35cba admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:29.339024 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d4b25a-f417-43b5-9b35-4351a8209dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:29.339636 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d4b25a-f417-43b5-9b35-4351a8209dd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:29.339918 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d4b25a-f417-43b5-9b35-4351a8209dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:29.340193 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d4b25a-f417-43b5-9b35-4351a8209dd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:29.340896 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d4b25a-f417-43b5-9b35-4351a8209dd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:29.342511 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d4b25a-f417-43b5-9b35-4351a8209dd5 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:29.353965 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1082/2178] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:14:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:14:29.358090 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae6f109-aa94-4071-b0b7-3b3637cd868a admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:29.363844 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:29.369363 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae6f109-aa94-4071-b0b7-3b3637cd868a admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:29.374351 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1061da-735b-45b8-9218-e4429c7c62a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:29.375070 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1061da-735b-45b8-9218-e4429c7c62a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:29.375456 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1061da-735b-45b8-9218-e4429c7c62a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:29.375790 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1061da-735b-45b8-9218-e4429c7c62a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:29.376501 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1061da-735b-45b8-9218-e4429c7c62a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:29.378201 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1061da-735b-45b8-9218-e4429c7c62a6 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:29.390502 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1097/2179] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:14:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:14:29.394720 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d4b25a-f417-43b5-9b35-4351a8209dd5 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:29.406033 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d4b25a-f417-43b5-9b35-4351a8209dd5 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:29.410750 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb028365-7b90-4717-9942-3ee341c59f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:29.411427 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb028365-7b90-4717-9942-3ee341c59f5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:29.411709 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb028365-7b90-4717-9942-3ee341c59f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:29.411983 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb028365-7b90-4717-9942-3ee341c59f5d admin admin] PATH_INFO: `/v3/projects/420292ae9c0547cc9755f24c23d419c3/users/32f58f87972d46be9496e6216f1f5ca1/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:29.427936 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb028365-7b90-4717-9942-3ee341c59f5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=420292ae9c0547cc9755f24c23d419c3, user_id=32f58f87972d46be9496e6216f1f5ca1, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:29.430998 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb028365-7b90-4717-9942-3ee341c59f5d admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:29.504614 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1083/2180] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:14:29 2022] PUT /identity/v3/projects/420292ae9c0547cc9755f24c23d419c3/users/32f58f87972d46be9496e6216f1f5ca1/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:14:29.510039 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf87f5e-8bdb-4fcf-a43e-b3793f0f77f7 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:29.510433 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf87f5e-8bdb-4fcf-a43e-b3793f0f77f7 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:29.510809 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf87f5e-8bdb-4fcf-a43e-b3793f0f77f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:29.530620 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-fcf87f5e-8bdb-4fcf-a43e-b3793f0f77f7 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:14:29.531937 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-fcf87f5e-8bdb-4fcf-a43e-b3793f0f77f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:29.597887 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1098/2181] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:14:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:14:29.611928 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb028365-7b90-4717-9942-3ee341c59f5d admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:29.614433 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-bb028365-7b90-4717-9942-3ee341c59f5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:29.695091 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb028365-7b90-4717-9942-3ee341c59f5d admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:29.698643 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82eee62-6f34-45fc-9be3-0e11b1413e24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:29.699292 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82eee62-6f34-45fc-9be3-0e11b1413e24 service neutron] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:29.699577 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82eee62-6f34-45fc-9be3-0e11b1413e24 service neutron] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:29.699855 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82eee62-6f34-45fc-9be3-0e11b1413e24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:29.710195 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82eee62-6f34-45fc-9be3-0e11b1413e24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:29.711887 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82eee62-6f34-45fc-9be3-0e11b1413e24 service neutron] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:29.722276 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1084/2182] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:14:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:14:31.065169 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce25c523-f3fa-46c8-8f56-2d1e3c98b82f None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:31.066582 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce25c523-f3fa-46c8-8f56-2d1e3c98b82f None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:31.066936 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce25c523-f3fa-46c8-8f56-2d1e3c98b82f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:31.103970 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-ce25c523-f3fa-46c8-8f56-2d1e3c98b82f None None] MFA Rules not processed for user `32f58f87972d46be9496e6216f1f5ca1`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:14:31.105230 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-ce25c523-f3fa-46c8-8f56-2d1e3c98b82f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:31.205842 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1099/2183] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:14:31 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:14:31.219349 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82eee62-6f34-45fc-9be3-0e11b1413e24 service neutron] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:31.221324 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-d82eee62-6f34-45fc-9be3-0e11b1413e24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:31.254069 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:31.293616 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82eee62-6f34-45fc-9be3-0e11b1413e24 service neutron] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:31.300323 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0573fe96-50ce-4f5b-8d7d-42b16712bfce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:31.301149 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0573fe96-50ce-4f5b-8d7d-42b16712bfce service cinder] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:31.301581 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0573fe96-50ce-4f5b-8d7d-42b16712bfce service cinder] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:31.302005 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0573fe96-50ce-4f5b-8d7d-42b16712bfce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:31.316469 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0573fe96-50ce-4f5b-8d7d-42b16712bfce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:31.318159 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0573fe96-50ce-4f5b-8d7d-42b16712bfce service cinder] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:31.336955 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1085/2184] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:14:31 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:14:31.905313 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d1089-5945-4201-9871-cb797ad3ad02 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:31.905611 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d1089-5945-4201-9871-cb797ad3ad02 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:31.905901 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d1089-5945-4201-9871-cb797ad3ad02 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:31.908312 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1100/2185] 10.222.0.82 () {54 vars in 921 bytes} [Thu Aug 18 17:14:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:14:31.919534 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c3b6e-956d-4325-af2a-c8b5debf7efc None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:31.926156 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c3b6e-956d-4325-af2a-c8b5debf7efc None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:31.926156 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c3b6e-956d-4325-af2a-c8b5debf7efc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:31.954065 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-099c3b6e-956d-4325-af2a-c8b5debf7efc None None] MFA Rules not processed for user `29653ba1a2a74232972ef7a9000518b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:14:31.955409 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-099c3b6e-956d-4325-af2a-c8b5debf7efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:32.033588 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1086/2186] 10.222.0.82 () {58 vars in 1018 bytes} [Thu Aug 18 17:14:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:14:32.157337 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8d1089-5945-4201-9871-cb797ad3ad02 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:32.159738 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-5e8d1089-5945-4201-9871-cb797ad3ad02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:32.183192 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8d1089-5945-4201-9871-cb797ad3ad02 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:32.188394 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb037a81-62c1-448a-af41-4cd4e02be024 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29653ba1a2a74232972ef7a9000518b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:32.189175 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb037a81-62c1-448a-af41-4cd4e02be024 service placement] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:32.189467 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb037a81-62c1-448a-af41-4cd4e02be024 service placement] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:32.189749 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb037a81-62c1-448a-af41-4cd4e02be024 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:32.198692 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb037a81-62c1-448a-af41-4cd4e02be024 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:32.203206 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb037a81-62c1-448a-af41-4cd4e02be024 service placement] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:32.210450 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1101/2187] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:14:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:14:32.290219 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099c3b6e-956d-4325-af2a-c8b5debf7efc None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:32.291745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-099c3b6e-956d-4325-af2a-c8b5debf7efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:32.306652 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099c3b6e-956d-4325-af2a-c8b5debf7efc None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:32.311280 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a10a4b0-7258-403e-9929-8f9ef4efbd5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:32.311911 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a10a4b0-7258-403e-9929-8f9ef4efbd5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:32.312193 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a10a4b0-7258-403e-9929-8f9ef4efbd5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:32.312469 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a10a4b0-7258-403e-9929-8f9ef4efbd5e admin admin] PATH_INFO: `/v3/users/e72104f038d24821be824a45e108d1ac` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:32.313600 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a10a4b0-7258-403e-9929-8f9ef4efbd5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e72104f038d24821be824a45e108d1ac)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:32.315729 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a10a4b0-7258-403e-9929-8f9ef4efbd5e admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:32.511861 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-8a10a4b0-7258-403e-9929-8f9ef4efbd5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e72104f038d24821be824a45e108d1ac'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:32.524135 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-8a10a4b0-7258-403e-9929-8f9ef4efbd5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e72104f038d24821be824a45e108d1ac'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:32.526839 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-8a10a4b0-7258-403e-9929-8f9ef4efbd5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e72104f038d24821be824a45e108d1ac'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:32.601173 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-8a10a4b0-7258-403e-9929-8f9ef4efbd5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e72104f038d24821be824a45e108d1ac'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:32.602593 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-8a10a4b0-7258-403e-9929-8f9ef4efbd5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e72104f038d24821be824a45e108d1ac'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:32.616914 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1087/2188] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:14:32 2022] DELETE /identity/v3/users/e72104f038d24821be824a45e108d1ac => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:14:32.848053 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb037a81-62c1-448a-af41-4cd4e02be024 service placement] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:32.859352 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb037a81-62c1-448a-af41-4cd4e02be024 service placement] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:32.917832 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f230847-0085-4870-a71c-7435e59cd82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:32.918480 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f230847-0085-4870-a71c-7435e59cd82f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:32.918765 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f230847-0085-4870-a71c-7435e59cd82f admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:32.919070 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f230847-0085-4870-a71c-7435e59cd82f admin admin] PATH_INFO: `/v3/projects/66c908d398424970b13440ee7f6048d6` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:32.920226 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f230847-0085-4870-a71c-7435e59cd82f admin admin] RBAC: Authorizing `identity:delete_project(project_id=66c908d398424970b13440ee7f6048d6)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:32.922117 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f230847-0085-4870-a71c-7435e59cd82f admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:32.933266 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:33.046430 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-1f230847-0085-4870-a71c-7435e59cd82f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66c908d398424970b13440ee7f6048d6', 'request_id': 'req-1f230847-0085-4870-a71c-7435e59cd82f'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:33.053765 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-1f230847-0085-4870-a71c-7435e59cd82f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66c908d398424970b13440ee7f6048d6', 'request_id': 'req-1f230847-0085-4870-a71c-7435e59cd82f'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:33.138453 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-1f230847-0085-4870-a71c-7435e59cd82f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66c908d398424970b13440ee7f6048d6', 'request_id': 'req-1f230847-0085-4870-a71c-7435e59cd82f'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:33.151514 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-1f230847-0085-4870-a71c-7435e59cd82f admin admin] The token cache is being invalidate because project 66c908d398424970b13440ee7f6048d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:14:33.151872 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-1f230847-0085-4870-a71c-7435e59cd82f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:33.152670 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-1f230847-0085-4870-a71c-7435e59cd82f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:33.154226 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1102/2189] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:14:32 2022] DELETE /identity/v3/projects/66c908d398424970b13440ee7f6048d6 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:14:33.159065 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c616b40-f411-4521-a7d5-644baa805e5e None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:33.159454 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c616b40-f411-4521-a7d5-644baa805e5e None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:33.159787 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c616b40-f411-4521-a7d5-644baa805e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:33.180564 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-7c616b40-f411-4521-a7d5-644baa805e5e None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:14:33.181773 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-7c616b40-f411-4521-a7d5-644baa805e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:33.207414 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:33.239289 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1088/2190] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:14:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:14:33.247939 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e3eea-b98f-4954-83f9-0cd4cc40cab5 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:33.248429 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e3eea-b98f-4954-83f9-0cd4cc40cab5 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:33.248649 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e3eea-b98f-4954-83f9-0cd4cc40cab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:33.268574 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-896e3eea-b98f-4954-83f9-0cd4cc40cab5 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:14:33.269761 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-896e3eea-b98f-4954-83f9-0cd4cc40cab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:33.277752 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1103/2191] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:14:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:14:33.282128 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c616b40-f411-4521-a7d5-644baa805e5e None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:33.292178 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c616b40-f411-4521-a7d5-644baa805e5e None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:33.296694 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaef390-e9cf-46bb-9311-41f13a35ba13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:33.297288 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaef390-e9cf-46bb-9311-41f13a35ba13 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:33.297567 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaef390-e9cf-46bb-9311-41f13a35ba13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:33.297870 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaef390-e9cf-46bb-9311-41f13a35ba13 admin admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:33.298471 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaef390-e9cf-46bb-9311-41f13a35ba13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:14:33.298770 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaef390-e9cf-46bb-9311-41f13a35ba13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:33.300400 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaef390-e9cf-46bb-9311-41f13a35ba13 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:33.312111 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1089/2192] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:14:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 17:14:33.316731 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896e3eea-b98f-4954-83f9-0cd4cc40cab5 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:33.327307 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896e3eea-b98f-4954-83f9-0cd4cc40cab5 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:33.332391 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc8651c-604b-481c-9a42-f2b359934819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:33.337254 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc8651c-604b-481c-9a42-f2b359934819 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:33.337254 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc8651c-604b-481c-9a42-f2b359934819 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:33.337254 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc8651c-604b-481c-9a42-f2b359934819 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:33.337254 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc8651c-604b-481c-9a42-f2b359934819 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:33.337254 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc8651c-604b-481c-9a42-f2b359934819 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:33.480747 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1104/2193] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:14:33 2022] POST /identity/v3/projects => generated 367 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:14:33.484818 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaef390-e9cf-46bb-9311-41f13a35ba13 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:33.495766 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaef390-e9cf-46bb-9311-41f13a35ba13 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:33.564607 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c04920-700e-46c6-945f-a82959baa4b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:33.565279 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c04920-700e-46c6-945f-a82959baa4b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:33.565568 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c04920-700e-46c6-945f-a82959baa4b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:33.565842 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c04920-700e-46c6-945f-a82959baa4b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:33.566461 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c04920-700e-46c6-945f-a82959baa4b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:33.568667 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c04920-700e-46c6-945f-a82959baa4b7 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:33.670190 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1090/2194] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:14:33 2022] POST /identity/v3/users => generated 284 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:14:33.672474 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc8651c-604b-481c-9a42-f2b359934819 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:33.684303 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc8651c-604b-481c-9a42-f2b359934819 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:33.689463 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24918c99-1bfc-4ff0-8277-a85af00b60c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:33.690164 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24918c99-1bfc-4ff0-8277-a85af00b60c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:33.690529 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24918c99-1bfc-4ff0-8277-a85af00b60c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:33.690934 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24918c99-1bfc-4ff0-8277-a85af00b60c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:33.691694 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24918c99-1bfc-4ff0-8277-a85af00b60c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:33.693418 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24918c99-1bfc-4ff0-8277-a85af00b60c6 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:33.704820 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1105/2195] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:14:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 17:14:33.709030 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c04920-700e-46c6-945f-a82959baa4b7 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:33.719858 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c04920-700e-46c6-945f-a82959baa4b7 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:33.724647 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c444e85-40af-4d0b-8d8d-c71ce501d041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:33.725756 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c444e85-40af-4d0b-8d8d-c71ce501d041 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:33.727348 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c444e85-40af-4d0b-8d8d-c71ce501d041 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:33.727348 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c444e85-40af-4d0b-8d8d-c71ce501d041 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:33.727348 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c444e85-40af-4d0b-8d8d-c71ce501d041 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:33.728572 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c444e85-40af-4d0b-8d8d-c71ce501d041 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:33.739626 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1091/2196] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:14:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:14:33.744349 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24918c99-1bfc-4ff0-8277-a85af00b60c6 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:33.754966 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24918c99-1bfc-4ff0-8277-a85af00b60c6 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:33.760258 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e1a679-ed7e-4d51-b7cf-a67c7524d221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:33.760951 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e1a679-ed7e-4d51-b7cf-a67c7524d221 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:33.761238 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e1a679-ed7e-4d51-b7cf-a67c7524d221 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:33.761516 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e1a679-ed7e-4d51-b7cf-a67c7524d221 admin admin] PATH_INFO: `/v3/projects/c60b160a12ec4c3ebc8f8515d88e36db/users/030f1b12af694e168efdfbc79f156931/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:33.777730 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e1a679-ed7e-4d51-b7cf-a67c7524d221 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c60b160a12ec4c3ebc8f8515d88e36db, user_id=030f1b12af694e168efdfbc79f156931, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:33.780518 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e1a679-ed7e-4d51-b7cf-a67c7524d221 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:33.871296 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1106/2197] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:14:33 2022] PUT /identity/v3/projects/c60b160a12ec4c3ebc8f8515d88e36db/users/030f1b12af694e168efdfbc79f156931/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:14:33.881582 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a6151c-53e3-45b5-953e-26c110f6fca4 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:33.881878 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a6151c-53e3-45b5-953e-26c110f6fca4 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:33.882484 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a6151c-53e3-45b5-953e-26c110f6fca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:33.911536 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-49a6151c-53e3-45b5-953e-26c110f6fca4 None None] MFA Rules not processed for user `030f1b12af694e168efdfbc79f156931`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:14:33.913412 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-49a6151c-53e3-45b5-953e-26c110f6fca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:33.978938 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1092/2198] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:14:33 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:14:33.983693 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e1a679-ed7e-4d51-b7cf-a67c7524d221 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:34.049764 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e1a679-ed7e-4d51-b7cf-a67c7524d221 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:34.055049 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8195cdbb-b67c-4441-b62e-9d1420d190a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:34.055665 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8195cdbb-b67c-4441-b62e-9d1420d190a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:34.055950 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8195cdbb-b67c-4441-b62e-9d1420d190a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:34.056227 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8195cdbb-b67c-4441-b62e-9d1420d190a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:34.056900 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8195cdbb-b67c-4441-b62e-9d1420d190a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:34.058842 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8195cdbb-b67c-4441-b62e-9d1420d190a7 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:34.165141 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1107/2199] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:14:33 2022] POST /identity/v3/projects => generated 367 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:14:34.169314 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a6151c-53e3-45b5-953e-26c110f6fca4 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:34.184321 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a6151c-53e3-45b5-953e-26c110f6fca4 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:34.219014 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba53463-1515-4f92-8ab5-1f7b4e861793 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:34.219553 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba53463-1515-4f92-8ab5-1f7b4e861793 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:34.219935 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba53463-1515-4f92-8ab5-1f7b4e861793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:34.231295 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:34.246403 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-dba53463-1515-4f92-8ab5-1f7b4e861793 None None] MFA Rules not processed for user `32f58f87972d46be9496e6216f1f5ca1`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:14:34.247827 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-dba53463-1515-4f92-8ab5-1f7b4e861793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:34.267864 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:34.273755 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159551c4-d3aa-4413-973f-15939a5b6cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:34.274382 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159551c4-d3aa-4413-973f-15939a5b6cc9 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:34.274665 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159551c4-d3aa-4413-973f-15939a5b6cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:34.274983 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159551c4-d3aa-4413-973f-15939a5b6cc9 admin admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:34.276254 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159551c4-d3aa-4413-973f-15939a5b6cc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:34.277581 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159551c4-d3aa-4413-973f-15939a5b6cc9 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:34.319933 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1108/2200] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:14:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 17:14:34.387962 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1093/2201] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:14:34 2022] POST /identity/v3/users => generated 290 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 17:14:34.389610 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba53463-1515-4f92-8ab5-1f7b4e861793 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:34.401353 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba53463-1515-4f92-8ab5-1f7b4e861793 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:34.406235 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86619ef4-4388-4ddc-aac6-9c555b9df15a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:34.406857 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86619ef4-4388-4ddc-aac6-9c555b9df15a admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:34.411001 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86619ef4-4388-4ddc-aac6-9c555b9df15a admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:34.411001 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86619ef4-4388-4ddc-aac6-9c555b9df15a admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:34.411001 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86619ef4-4388-4ddc-aac6-9c555b9df15a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:34.411001 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86619ef4-4388-4ddc-aac6-9c555b9df15a admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:34.420249 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1109/2202] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:14:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:14:34.424339 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159551c4-d3aa-4413-973f-15939a5b6cc9 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:34.434585 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159551c4-d3aa-4413-973f-15939a5b6cc9 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:34.439913 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561dacb3-d298-468f-b4c0-bd387929f5a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:34.440580 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561dacb3-d298-468f-b4c0-bd387929f5a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:34.440900 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561dacb3-d298-468f-b4c0-bd387929f5a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:34.441217 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561dacb3-d298-468f-b4c0-bd387929f5a3 admin admin] PATH_INFO: `/v3/projects/7ac93704e4a34c95babf0f9ff5e76a37/users/42132a4670d446888fc2985c409705b9/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:34.456987 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561dacb3-d298-468f-b4c0-bd387929f5a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ac93704e4a34c95babf0f9ff5e76a37, user_id=42132a4670d446888fc2985c409705b9, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:34.460380 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561dacb3-d298-468f-b4c0-bd387929f5a3 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:34.539748 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1094/2203] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:14:34 2022] PUT /identity/v3/projects/7ac93704e4a34c95babf0f9ff5e76a37/users/42132a4670d446888fc2985c409705b9/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:14:34.543794 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86619ef4-4388-4ddc-aac6-9c555b9df15a admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:34.554635 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86619ef4-4388-4ddc-aac6-9c555b9df15a admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:34.617688 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b9bd3-22e8-4ad2-84d9-975ac719b723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:34.618419 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b9bd3-22e8-4ad2-84d9-975ac719b723 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:34.618792 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b9bd3-22e8-4ad2-84d9-975ac719b723 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:34.619335 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b9bd3-22e8-4ad2-84d9-975ac719b723 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:34.620089 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861b9bd3-22e8-4ad2-84d9-975ac719b723 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:34.621598 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861b9bd3-22e8-4ad2-84d9-975ac719b723 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:34.633056 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1110/2204] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:14:34 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 17:14:34.638353 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561dacb3-d298-468f-b4c0-bd387929f5a3 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:34.651440 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561dacb3-d298-468f-b4c0-bd387929f5a3 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:34.656646 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb50794c-77bd-429a-a41c-702da452634b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:34.657339 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb50794c-77bd-429a-a41c-702da452634b admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:34.657722 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb50794c-77bd-429a-a41c-702da452634b admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:34.657963 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb50794c-77bd-429a-a41c-702da452634b admin admin] PATH_INFO: `/v3/projects/7ac93704e4a34c95babf0f9ff5e76a37/users/42132a4670d446888fc2985c409705b9/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:34.660202 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb50794c-77bd-429a-a41c-702da452634b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ac93704e4a34c95babf0f9ff5e76a37, user_id=42132a4670d446888fc2985c409705b9, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:34.663690 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb50794c-77bd-429a-a41c-702da452634b admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:34.675680 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1095/2205] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:14:34 2022] PUT /identity/v3/projects/7ac93704e4a34c95babf0f9ff5e76a37/users/42132a4670d446888fc2985c409705b9/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:14:34.688831 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d28a0de-6e87-47cf-aa4f-0d24a0146a01 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:34.708954 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d28a0de-6e87-47cf-aa4f-0d24a0146a01 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:34.708954 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d28a0de-6e87-47cf-aa4f-0d24a0146a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:34.722311 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-2d28a0de-6e87-47cf-aa4f-0d24a0146a01 None None] MFA Rules not processed for user `42132a4670d446888fc2985c409705b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:14:34.723604 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-2d28a0de-6e87-47cf-aa4f-0d24a0146a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:34.822946 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1111/2206] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:14:34 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:14:34.837100 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb50794c-77bd-429a-a41c-702da452634b admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:34.839132 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-cb50794c-77bd-429a-a41c-702da452634b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:34.954384 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb50794c-77bd-429a-a41c-702da452634b admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:34.955862 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd06876d-738a-4c14-8c2b-931d1cd2ddc6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:34.956563 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd06876d-738a-4c14-8c2b-931d1cd2ddc6 service cinder] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:34.956915 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd06876d-738a-4c14-8c2b-931d1cd2ddc6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:34.957262 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd06876d-738a-4c14-8c2b-931d1cd2ddc6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:34.967731 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd06876d-738a-4c14-8c2b-931d1cd2ddc6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:34.969320 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd06876d-738a-4c14-8c2b-931d1cd2ddc6 service cinder] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:34.982108 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1096/2207] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:14:34 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:14:35.091536 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d28a0de-6e87-47cf-aa4f-0d24a0146a01 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:35.113336 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d28a0de-6e87-47cf-aa4f-0d24a0146a01 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:35.120404 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c888062-41ee-4924-b2ca-30367cabfda7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:35.121162 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c888062-41ee-4924-b2ca-30367cabfda7 service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:35.121681 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c888062-41ee-4924-b2ca-30367cabfda7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:35.121929 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c888062-41ee-4924-b2ca-30367cabfda7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:35.161202 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:35.191843 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c888062-41ee-4924-b2ca-30367cabfda7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:35.195991 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c888062-41ee-4924-b2ca-30367cabfda7 service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:35.204152 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1112/2208] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:14:35 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:14:38.580795 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd06876d-738a-4c14-8c2b-931d1cd2ddc6 service cinder] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:38.582546 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-bd06876d-738a-4c14-8c2b-931d1cd2ddc6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:38.651056 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd06876d-738a-4c14-8c2b-931d1cd2ddc6 service cinder] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:38.655463 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12309dd2-e93f-48a0-b901-5cfa0d542e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:38.656091 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12309dd2-e93f-48a0-b901-5cfa0d542e28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:38.656368 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12309dd2-e93f-48a0-b901-5cfa0d542e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:38.656637 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12309dd2-e93f-48a0-b901-5cfa0d542e28 admin admin] PATH_INFO: `/v3/users/413962826e1843f68fc2bda00e67af18` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:38.657748 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12309dd2-e93f-48a0-b901-5cfa0d542e28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=413962826e1843f68fc2bda00e67af18)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:38.659708 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12309dd2-e93f-48a0-b901-5cfa0d542e28 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:38.684975 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:38.829339 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-12309dd2-e93f-48a0-b901-5cfa0d542e28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '413962826e1843f68fc2bda00e67af18'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:38.842259 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-12309dd2-e93f-48a0-b901-5cfa0d542e28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '413962826e1843f68fc2bda00e67af18'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:38.842945 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-12309dd2-e93f-48a0-b901-5cfa0d542e28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '413962826e1843f68fc2bda00e67af18'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:38.957653 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-12309dd2-e93f-48a0-b901-5cfa0d542e28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '413962826e1843f68fc2bda00e67af18'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:38.958191 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-12309dd2-e93f-48a0-b901-5cfa0d542e28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '413962826e1843f68fc2bda00e67af18'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:38.971776 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1097/2209] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:14:38 2022] DELETE /identity/v3/users/413962826e1843f68fc2bda00e67af18 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:14:38.976515 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75baf44-0fab-4eef-9cc3-540481143049 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:38.976985 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75baf44-0fab-4eef-9cc3-540481143049 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:38.977077 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75baf44-0fab-4eef-9cc3-540481143049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:38.996075 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-c75baf44-0fab-4eef-9cc3-540481143049 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:14:38.997300 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-c75baf44-0fab-4eef-9cc3-540481143049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:39.057542 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1113/2210] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:14:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 17:14:39.067378 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12309dd2-e93f-48a0-b901-5cfa0d542e28 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:39.069293 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-12309dd2-e93f-48a0-b901-5cfa0d542e28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:39.128102 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12309dd2-e93f-48a0-b901-5cfa0d542e28 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:39.132521 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95757dd-7cec-40e9-ae37-95d79c3609fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:39.133124 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95757dd-7cec-40e9-ae37-95d79c3609fd service neutron] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:39.133403 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95757dd-7cec-40e9-ae37-95d79c3609fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:39.133677 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95757dd-7cec-40e9-ae37-95d79c3609fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:39.142843 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95757dd-7cec-40e9-ae37-95d79c3609fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:39.144324 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95757dd-7cec-40e9-ae37-95d79c3609fd service neutron] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:39.153658 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1098/2211] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:14:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:14:39.634112 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75baf44-0fab-4eef-9cc3-540481143049 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:39.635869 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-c75baf44-0fab-4eef-9cc3-540481143049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:39.652355 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75baf44-0fab-4eef-9cc3-540481143049 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:39.660373 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2bc085-33c9-4c46-8202-6dc9ebef12e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:39.663613 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2bc085-33c9-4c46-8202-6dc9ebef12e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:39.663613 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2bc085-33c9-4c46-8202-6dc9ebef12e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:39.663613 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2bc085-33c9-4c46-8202-6dc9ebef12e8 admin admin] PATH_INFO: `/v3/projects/637180141b1944859c22bb909e376e10` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:39.663613 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2bc085-33c9-4c46-8202-6dc9ebef12e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=637180141b1944859c22bb909e376e10)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:39.666778 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2bc085-33c9-4c46-8202-6dc9ebef12e8 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:39.866167 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ce2bc085-33c9-4c46-8202-6dc9ebef12e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '637180141b1944859c22bb909e376e10', 'request_id': 'req-ce2bc085-33c9-4c46-8202-6dc9ebef12e8'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:39.881431 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ce2bc085-33c9-4c46-8202-6dc9ebef12e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '637180141b1944859c22bb909e376e10', 'request_id': 'req-ce2bc085-33c9-4c46-8202-6dc9ebef12e8'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:39.995866 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ce2bc085-33c9-4c46-8202-6dc9ebef12e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '637180141b1944859c22bb909e376e10', 'request_id': 'req-ce2bc085-33c9-4c46-8202-6dc9ebef12e8'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:40.019647 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ce2bc085-33c9-4c46-8202-6dc9ebef12e8 admin admin] The token cache is being invalidate because project 637180141b1944859c22bb909e376e10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:14:40.019943 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ce2bc085-33c9-4c46-8202-6dc9ebef12e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:40.020605 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ce2bc085-33c9-4c46-8202-6dc9ebef12e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:40.023347 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1114/2212] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:14:39 2022] DELETE /identity/v3/projects/637180141b1944859c22bb909e376e10 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:14:40.027251 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95757dd-7cec-40e9-ae37-95d79c3609fd service neutron] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:40.037880 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95757dd-7cec-40e9-ae37-95d79c3609fd service neutron] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:40.109318 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ecf269-6a60-476e-8250-c740082d3fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:40.110223 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ecf269-6a60-476e-8250-c740082d3fc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:40.110509 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ecf269-6a60-476e-8250-c740082d3fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:40.110778 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ecf269-6a60-476e-8250-c740082d3fc5 admin admin] PATH_INFO: `/v3/users/cdb883bd438a460ea89ad3bd277df2fe` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:40.112008 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ecf269-6a60-476e-8250-c740082d3fc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdb883bd438a460ea89ad3bd277df2fe)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:40.113958 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ecf269-6a60-476e-8250-c740082d3fc5 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:40.348039 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:40.349658 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-17ecf269-6a60-476e-8250-c740082d3fc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdb883bd438a460ea89ad3bd277df2fe'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:40.364405 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-17ecf269-6a60-476e-8250-c740082d3fc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdb883bd438a460ea89ad3bd277df2fe'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:40.365325 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-17ecf269-6a60-476e-8250-c740082d3fc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdb883bd438a460ea89ad3bd277df2fe'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:40.471975 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-17ecf269-6a60-476e-8250-c740082d3fc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdb883bd438a460ea89ad3bd277df2fe'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:40.472707 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-17ecf269-6a60-476e-8250-c740082d3fc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdb883bd438a460ea89ad3bd277df2fe'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:40.490740 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1099/2213] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:14:40 2022] DELETE /identity/v3/users/cdb883bd438a460ea89ad3bd277df2fe => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:14:40.718984 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2bc085-33c9-4c46-8202-6dc9ebef12e8 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:40.720653 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-ce2bc085-33c9-4c46-8202-6dc9ebef12e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:40.738451 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2bc085-33c9-4c46-8202-6dc9ebef12e8 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:40.843691 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:40.852244 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3defa58b-7ed7-49e0-9b04-d6e568b1a195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:40.854351 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3defa58b-7ed7-49e0-9b04-d6e568b1a195 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:40.854351 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3defa58b-7ed7-49e0-9b04-d6e568b1a195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:40.854351 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3defa58b-7ed7-49e0-9b04-d6e568b1a195 admin admin] PATH_INFO: `/v3/users/030f1b12af694e168efdfbc79f156931` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:40.855348 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3defa58b-7ed7-49e0-9b04-d6e568b1a195 admin admin] RBAC: Authorizing `identity:delete_user(user_id=030f1b12af694e168efdfbc79f156931)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:40.858010 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3defa58b-7ed7-49e0-9b04-d6e568b1a195 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:40.968527 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ecf269-6a60-476e-8250-c740082d3fc5 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:40.970483 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-17ecf269-6a60-476e-8250-c740082d3fc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:41.004864 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ecf269-6a60-476e-8250-c740082d3fc5 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:41.011583 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3a0e7c-81d2-41df-b3b7-072149acfd39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:41.012274 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a0e7c-81d2-41df-b3b7-072149acfd39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:41.012560 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a0e7c-81d2-41df-b3b7-072149acfd39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:41.012834 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a0e7c-81d2-41df-b3b7-072149acfd39 admin admin] PATH_INFO: `/v3/projects/66b0ad02ff55423582b2c69824416954` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:41.014096 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3a0e7c-81d2-41df-b3b7-072149acfd39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66b0ad02ff55423582b2c69824416954)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:41.017290 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3a0e7c-81d2-41df-b3b7-072149acfd39 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:41.130187 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-3defa58b-7ed7-49e0-9b04-d6e568b1a195 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '030f1b12af694e168efdfbc79f156931'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:41.201468 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-6d3a0e7c-81d2-41df-b3b7-072149acfd39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66b0ad02ff55423582b2c69824416954', 'request_id': 'req-6d3a0e7c-81d2-41df-b3b7-072149acfd39'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:41.229679 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-3defa58b-7ed7-49e0-9b04-d6e568b1a195 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '030f1b12af694e168efdfbc79f156931'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:41.232272 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-3defa58b-7ed7-49e0-9b04-d6e568b1a195 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '030f1b12af694e168efdfbc79f156931'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:41.232272 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-3defa58b-7ed7-49e0-9b04-d6e568b1a195 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '030f1b12af694e168efdfbc79f156931'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:41.244328 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-3defa58b-7ed7-49e0-9b04-d6e568b1a195 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '030f1b12af694e168efdfbc79f156931'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:41.289887 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1115/2214] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:14:40 2022] DELETE /identity/v3/users/030f1b12af694e168efdfbc79f156931 => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:14:41.299306 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e3cd06-b20a-4d17-ad9f-4ae3a4231a15 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:41.299925 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e3cd06-b20a-4d17-ad9f-4ae3a4231a15 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:41.301917 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e3cd06-b20a-4d17-ad9f-4ae3a4231a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:41.324461 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-b9e3cd06-b20a-4d17-ad9f-4ae3a4231a15 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:14:41.325746 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-b9e3cd06-b20a-4d17-ad9f-4ae3a4231a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:41.386890 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1116/2215] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:14:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:14:41.397866 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e3cd06-b20a-4d17-ad9f-4ae3a4231a15 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:41.399336 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-b9e3cd06-b20a-4d17-ad9f-4ae3a4231a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:41.418796 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-6d3a0e7c-81d2-41df-b3b7-072149acfd39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66b0ad02ff55423582b2c69824416954', 'request_id': 'req-6d3a0e7c-81d2-41df-b3b7-072149acfd39'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:41.426166 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-6d3a0e7c-81d2-41df-b3b7-072149acfd39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66b0ad02ff55423582b2c69824416954', 'request_id': 'req-6d3a0e7c-81d2-41df-b3b7-072149acfd39'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:41.450372 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-6d3a0e7c-81d2-41df-b3b7-072149acfd39 admin admin] The token cache is being invalidate because project 66b0ad02ff55423582b2c69824416954 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81650) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:14:41.450677 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-6d3a0e7c-81d2-41df-b3b7-072149acfd39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:41.451419 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-6d3a0e7c-81d2-41df-b3b7-072149acfd39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:41.452936 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1100/2216] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:14:40 2022] DELETE /identity/v3/projects/66b0ad02ff55423582b2c69824416954 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:14:41.457492 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7083689-f27e-4650-b902-107567e415c8 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:41.457780 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7083689-f27e-4650-b902-107567e415c8 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:41.458056 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7083689-f27e-4650-b902-107567e415c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:41.464582 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e3cd06-b20a-4d17-ad9f-4ae3a4231a15 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:41.468521 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec1770a-90d2-4867-8d32-a9ce189eb7d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:41.469184 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec1770a-90d2-4867-8d32-a9ce189eb7d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:41.469407 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec1770a-90d2-4867-8d32-a9ce189eb7d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:41.469686 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec1770a-90d2-4867-8d32-a9ce189eb7d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:41.471474 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-eec1770a-90d2-4867-8d32-a9ce189eb7d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:41.478757 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-e7083689-f27e-4650-b902-107567e415c8 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:14:41.479986 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-e7083689-f27e-4650-b902-107567e415c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:41.484851 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec1770a-90d2-4867-8d32-a9ce189eb7d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:41.486306 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec1770a-90d2-4867-8d32-a9ce189eb7d5 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:41.488004 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1101/2217] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:14:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:14:41.495754 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168280c5-6ef7-4719-b0b8-ae05e101dd8e None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:41.496040 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168280c5-6ef7-4719-b0b8-ae05e101dd8e None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:41.496479 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168280c5-6ef7-4719-b0b8-ae05e101dd8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:41.496619 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1117/2218] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:14:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 17:14:41.516791 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-168280c5-6ef7-4719-b0b8-ae05e101dd8e None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:14:41.518004 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-168280c5-6ef7-4719-b0b8-ae05e101dd8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:41.526715 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1102/2219] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:14:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 17:14:41.530799 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec1770a-90d2-4867-8d32-a9ce189eb7d5 service neutron] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:41.545778 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec1770a-90d2-4867-8d32-a9ce189eb7d5 service neutron] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:41.551998 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ec84c5-79ba-4a3d-970e-0cf75224a781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:41.552706 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec84c5-79ba-4a3d-970e-0cf75224a781 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:41.552984 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec84c5-79ba-4a3d-970e-0cf75224a781 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:41.553385 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec84c5-79ba-4a3d-970e-0cf75224a781 admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:41.554045 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ec84c5-79ba-4a3d-970e-0cf75224a781 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:14:41.554354 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ec84c5-79ba-4a3d-970e-0cf75224a781 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:41.556138 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ec84c5-79ba-4a3d-970e-0cf75224a781 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:41.569756 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1118/2220] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:14:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 17:14:41.574408 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168280c5-6ef7-4719-b0b8-ae05e101dd8e None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:41.584882 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168280c5-6ef7-4719-b0b8-ae05e101dd8e None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:41.590252 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8200e5-0c91-4057-91c9-f9d49356195d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:41.591096 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8200e5-0c91-4057-91c9-f9d49356195d admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:41.591433 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8200e5-0c91-4057-91c9-f9d49356195d admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:41.593453 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8200e5-0c91-4057-91c9-f9d49356195d admin admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:41.593453 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8200e5-0c91-4057-91c9-f9d49356195d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:41.594717 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8200e5-0c91-4057-91c9-f9d49356195d admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:41.709021 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1103/2221] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:14:41 2022] POST /identity/v3/projects => generated 357 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 17:14:41.715136 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ec84c5-79ba-4a3d-970e-0cf75224a781 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:41.727441 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ec84c5-79ba-4a3d-970e-0cf75224a781 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:41.793384 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f1a781-f5cc-4acf-ae62-0882985926f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:41.794038 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f1a781-f5cc-4acf-ae62-0882985926f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:41.794320 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f1a781-f5cc-4acf-ae62-0882985926f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:41.794595 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f1a781-f5cc-4acf-ae62-0882985926f5 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:41.795259 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f1a781-f5cc-4acf-ae62-0882985926f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:41.797187 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f1a781-f5cc-4acf-ae62-0882985926f5 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:41.803019 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:41.915873 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1119/2222] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:14:41 2022] POST /identity/v3/users => generated 279 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:14:41.920125 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8200e5-0c91-4057-91c9-f9d49356195d admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:41.930863 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8200e5-0c91-4057-91c9-f9d49356195d admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:41.935530 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bd61f1-7cf3-4611-ac24-d6fda055c87c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:41.936130 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd61f1-7cf3-4611-ac24-d6fda055c87c admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:41.936446 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd61f1-7cf3-4611-ac24-d6fda055c87c admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:41.936726 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd61f1-7cf3-4611-ac24-d6fda055c87c admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:41.937356 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bd61f1-7cf3-4611-ac24-d6fda055c87c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:41.939010 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bd61f1-7cf3-4611-ac24-d6fda055c87c admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:41.952307 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1104/2223] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:14:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:14:41.956311 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f1a781-f5cc-4acf-ae62-0882985926f5 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:41.967794 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f1a781-f5cc-4acf-ae62-0882985926f5 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:41.972753 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0700bcaf-270b-4bb3-82e2-dd39fea963ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:41.973511 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0700bcaf-270b-4bb3-82e2-dd39fea963ed admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:41.973795 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0700bcaf-270b-4bb3-82e2-dd39fea963ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:41.974072 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0700bcaf-270b-4bb3-82e2-dd39fea963ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:41.974762 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0700bcaf-270b-4bb3-82e2-dd39fea963ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:41.976407 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0700bcaf-270b-4bb3-82e2-dd39fea963ed admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:41.987699 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1120/2224] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:14:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:14:41.992633 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bd61f1-7cf3-4611-ac24-d6fda055c87c admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:42.002783 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bd61f1-7cf3-4611-ac24-d6fda055c87c admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:42.008546 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635b8c49-4741-48bc-888a-e5ae4adebacb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:42.009212 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635b8c49-4741-48bc-888a-e5ae4adebacb admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:42.009500 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635b8c49-4741-48bc-888a-e5ae4adebacb admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:42.009794 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635b8c49-4741-48bc-888a-e5ae4adebacb admin admin] PATH_INFO: `/v3/projects/9b85847639bf4d80b2fff671fba22e6c/users/4eab2fcc430b4300b31e480b65d82ac3/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:42.026399 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635b8c49-4741-48bc-888a-e5ae4adebacb admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b85847639bf4d80b2fff671fba22e6c, user_id=4eab2fcc430b4300b31e480b65d82ac3, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:42.029322 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635b8c49-4741-48bc-888a-e5ae4adebacb admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:42.034589 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0700bcaf-270b-4bb3-82e2-dd39fea963ed admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:42.036428 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-0700bcaf-270b-4bb3-82e2-dd39fea963ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:42.049582 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0700bcaf-270b-4bb3-82e2-dd39fea963ed admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:42.055391 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37baff51-6880-43fa-a36b-0955becfb753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:42.057139 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37baff51-6880-43fa-a36b-0955becfb753 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:42.063137 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37baff51-6880-43fa-a36b-0955becfb753 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:42.063137 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37baff51-6880-43fa-a36b-0955becfb753 admin admin] PATH_INFO: `/v3/projects/c60b160a12ec4c3ebc8f8515d88e36db` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:42.063137 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37baff51-6880-43fa-a36b-0955becfb753 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c60b160a12ec4c3ebc8f8515d88e36db)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:42.063137 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37baff51-6880-43fa-a36b-0955becfb753 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:42.118652 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1105/2225] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:14:41 2022] PUT /identity/v3/projects/9b85847639bf4d80b2fff671fba22e6c/users/4eab2fcc430b4300b31e480b65d82ac3/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:14:42.128634 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb1ea0-dca9-43fc-8cb5-9bb1a46fca58 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:42.128922 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb1ea0-dca9-43fc-8cb5-9bb1a46fca58 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:42.129198 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb1ea0-dca9-43fc-8cb5-9bb1a46fca58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:42.156403 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-37cb1ea0-dca9-43fc-8cb5-9bb1a46fca58 None None] MFA Rules not processed for user `4eab2fcc430b4300b31e480b65d82ac3`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:14:42.157595 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-37cb1ea0-dca9-43fc-8cb5-9bb1a46fca58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:42.181828 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:42.214662 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-37baff51-6880-43fa-a36b-0955becfb753 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c60b160a12ec4c3ebc8f8515d88e36db', 'request_id': 'req-37baff51-6880-43fa-a36b-0955becfb753'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:42.220924 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-37baff51-6880-43fa-a36b-0955becfb753 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c60b160a12ec4c3ebc8f8515d88e36db', 'request_id': 'req-37baff51-6880-43fa-a36b-0955becfb753'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:42.221831 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1106/2226] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:14:42 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 17:14:42.226494 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cb1ea0-dca9-43fc-8cb5-9bb1a46fca58 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:42.292486 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cb1ea0-dca9-43fc-8cb5-9bb1a46fca58 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:42.344073 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-37baff51-6880-43fa-a36b-0955becfb753 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c60b160a12ec4c3ebc8f8515d88e36db', 'request_id': 'req-37baff51-6880-43fa-a36b-0955becfb753'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:42.356507 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-37baff51-6880-43fa-a36b-0955becfb753 admin admin] The token cache is being invalidate because project c60b160a12ec4c3ebc8f8515d88e36db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:14:42.357220 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-37baff51-6880-43fa-a36b-0955becfb753 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:42.358084 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-37baff51-6880-43fa-a36b-0955becfb753 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:42.359070 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c072d3-3669-4563-b093-19c90c13cd68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:42.359857 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c072d3-3669-4563-b093-19c90c13cd68 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:42.360350 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c072d3-3669-4563-b093-19c90c13cd68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:42.360564 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1121/2227] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:14:42 2022] DELETE /identity/v3/projects/c60b160a12ec4c3ebc8f8515d88e36db => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:14:42.360969 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c072d3-3669-4563-b093-19c90c13cd68 admin admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:42.361684 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c072d3-3669-4563-b093-19c90c13cd68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:42.363756 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c072d3-3669-4563-b093-19c90c13cd68 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:42.364633 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37baff51-6880-43fa-a36b-0955becfb753 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:42.367049 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-37baff51-6880-43fa-a36b-0955becfb753 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:42.381518 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37baff51-6880-43fa-a36b-0955becfb753 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:42.387189 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3a016e-385d-4429-adc0-bfeceb0d9188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:42.387854 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a016e-385d-4429-adc0-bfeceb0d9188 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:42.388135 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a016e-385d-4429-adc0-bfeceb0d9188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:42.388407 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a016e-385d-4429-adc0-bfeceb0d9188 admin admin] PATH_INFO: `/v3/users/42132a4670d446888fc2985c409705b9` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:42.389589 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3a016e-385d-4429-adc0-bfeceb0d9188 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42132a4670d446888fc2985c409705b9)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:42.391606 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3a016e-385d-4429-adc0-bfeceb0d9188 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:42.502366 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1107/2228] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:14:42 2022] POST /identity/v3/projects => generated 355 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 17:14:42.506474 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c072d3-3669-4563-b093-19c90c13cd68 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:42.508273 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-13c072d3-3669-4563-b093-19c90c13cd68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:42.582893 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c072d3-3669-4563-b093-19c90c13cd68 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:42.588386 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b0d7eb-c679-4c2a-87ce-8bcc259a8855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:42.588990 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b0d7eb-c679-4c2a-87ce-8bcc259a8855 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:42.589703 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b0d7eb-c679-4c2a-87ce-8bcc259a8855 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:42.589703 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b0d7eb-c679-4c2a-87ce-8bcc259a8855 admin admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:42.590132 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b0d7eb-c679-4c2a-87ce-8bcc259a8855 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:42.592129 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b0d7eb-c679-4c2a-87ce-8bcc259a8855 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:42.642920 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-3e3a016e-385d-4429-adc0-bfeceb0d9188 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42132a4670d446888fc2985c409705b9'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:42.717729 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1108/2229] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:14:42 2022] POST /identity/v3/users => generated 284 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 17:14:42.721479 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b0d7eb-c679-4c2a-87ce-8bcc259a8855 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:42.731778 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b0d7eb-c679-4c2a-87ce-8bcc259a8855 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:42.734521 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-3e3a016e-385d-4429-adc0-bfeceb0d9188 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42132a4670d446888fc2985c409705b9'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:42.735157 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-3e3a016e-385d-4429-adc0-bfeceb0d9188 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42132a4670d446888fc2985c409705b9'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:42.735750 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-3e3a016e-385d-4429-adc0-bfeceb0d9188 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42132a4670d446888fc2985c409705b9'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:42.737705 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8fb201-45d1-40bc-a576-870653dff3d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:42.738675 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8fb201-45d1-40bc-a576-870653dff3d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:42.738994 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8fb201-45d1-40bc-a576-870653dff3d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:42.739445 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8fb201-45d1-40bc-a576-870653dff3d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:42.740766 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8fb201-45d1-40bc-a576-870653dff3d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:42.742404 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8fb201-45d1-40bc-a576-870653dff3d3 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:42.744196 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-3e3a016e-385d-4429-adc0-bfeceb0d9188 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42132a4670d446888fc2985c409705b9'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:42.745301 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=80,keystone:INSERT=5,keystone:DELETE=15 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:42.753250 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1109/2230] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 17:14:42.757557 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8fb201-45d1-40bc-a576-870653dff3d3 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:42.759452 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-6c8fb201-45d1-40bc-a576-870653dff3d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:42.761648 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1122/2231] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:14:42 2022] DELETE /identity/v3/users/42132a4670d446888fc2985c409705b9 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:14:42.774350 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8fb201-45d1-40bc-a576-870653dff3d3 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:42.812328 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:42.836717 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a594fa0-7ccb-42d1-81d3-fa49e2692491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:42.837394 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a594fa0-7ccb-42d1-81d3-fa49e2692491 admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:42.837681 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a594fa0-7ccb-42d1-81d3-fa49e2692491 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:42.837958 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a594fa0-7ccb-42d1-81d3-fa49e2692491 admin admin] PATH_INFO: `/v3/projects/3b97f3d88ec94b88907e89e4f7e1edaa/users/6021ae72beae4c939a13fbd2b62eaf5c/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:42.853624 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a594fa0-7ccb-42d1-81d3-fa49e2692491 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b97f3d88ec94b88907e89e4f7e1edaa, user_id=6021ae72beae4c939a13fbd2b62eaf5c, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:42.856353 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a594fa0-7ccb-42d1-81d3-fa49e2692491 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:42.932691 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1110/2232] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:14:42 2022] PUT /identity/v3/projects/3b97f3d88ec94b88907e89e4f7e1edaa/users/6021ae72beae4c939a13fbd2b62eaf5c/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:14:42.936842 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3a016e-385d-4429-adc0-bfeceb0d9188 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:42.947735 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3a016e-385d-4429-adc0-bfeceb0d9188 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:43.007695 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e39f37c-abef-4011-b315-db61d71b4c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:43.008318 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e39f37c-abef-4011-b315-db61d71b4c93 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:43.008600 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e39f37c-abef-4011-b315-db61d71b4c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:43.008876 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e39f37c-abef-4011-b315-db61d71b4c93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:43.009580 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e39f37c-abef-4011-b315-db61d71b4c93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:43.011196 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e39f37c-abef-4011-b315-db61d71b4c93 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:43.021478 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1123/2233] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:14:43.025615 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a594fa0-7ccb-42d1-81d3-fa49e2692491 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:43.035943 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a594fa0-7ccb-42d1-81d3-fa49e2692491 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:43.041131 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f9da87-a9f0-480f-9fbc-2bd08c37a225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:43.041946 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9da87-a9f0-480f-9fbc-2bd08c37a225 admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:43.042227 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9da87-a9f0-480f-9fbc-2bd08c37a225 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:43.042499 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9da87-a9f0-480f-9fbc-2bd08c37a225 admin admin] PATH_INFO: `/v3/projects/3b97f3d88ec94b88907e89e4f7e1edaa/users/6021ae72beae4c939a13fbd2b62eaf5c/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:43.044404 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f9da87-a9f0-480f-9fbc-2bd08c37a225 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b97f3d88ec94b88907e89e4f7e1edaa, user_id=6021ae72beae4c939a13fbd2b62eaf5c, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:43.047202 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f9da87-a9f0-480f-9fbc-2bd08c37a225 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:43.058187 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1111/2234] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:14:43 2022] PUT /identity/v3/projects/3b97f3d88ec94b88907e89e4f7e1edaa/users/6021ae72beae4c939a13fbd2b62eaf5c/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:14:43.070411 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d3a40-a593-4135-9f77-cd4978905353 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:43.070704 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d3a40-a593-4135-9f77-cd4978905353 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:43.071010 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d3a40-a593-4135-9f77-cd4978905353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:43.101523 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-d57d3a40-a593-4135-9f77-cd4978905353 None None] MFA Rules not processed for user `6021ae72beae4c939a13fbd2b62eaf5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:14:43.103064 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-d57d3a40-a593-4135-9f77-cd4978905353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:43.172823 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1124/2235] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:14:43 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:14:43.184908 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f9da87-a9f0-480f-9fbc-2bd08c37a225 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:43.186811 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-e5f9da87-a9f0-480f-9fbc-2bd08c37a225 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:43.199982 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57d3a40-a593-4135-9f77-cd4978905353 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:43.202035 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-d57d3a40-a593-4135-9f77-cd4978905353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:43.257891 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f9da87-a9f0-480f-9fbc-2bd08c37a225 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:43.261621 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a68f669-311c-4e7d-aa1e-9cef53b1271d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:43.262227 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a68f669-311c-4e7d-aa1e-9cef53b1271d service cinder] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:43.262509 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a68f669-311c-4e7d-aa1e-9cef53b1271d service cinder] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:43.262783 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a68f669-311c-4e7d-aa1e-9cef53b1271d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:43.272930 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57d3a40-a593-4135-9f77-cd4978905353 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:43.273400 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a68f669-311c-4e7d-aa1e-9cef53b1271d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:43.274761 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a68f669-311c-4e7d-aa1e-9cef53b1271d service cinder] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:43.277628 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17757dff-2115-45f1-98cc-d6736328b7e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:43.278551 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17757dff-2115-45f1-98cc-d6736328b7e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:43.278743 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17757dff-2115-45f1-98cc-d6736328b7e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:43.278943 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17757dff-2115-45f1-98cc-d6736328b7e4 admin admin] PATH_INFO: `/v3/projects/7ac93704e4a34c95babf0f9ff5e76a37` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:43.280114 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17757dff-2115-45f1-98cc-d6736328b7e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ac93704e4a34c95babf0f9ff5e76a37)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:43.282031 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17757dff-2115-45f1-98cc-d6736328b7e4 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:43.284737 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:43.286470 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1112/2236] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:14:43 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:14:43.431414 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-17757dff-2115-45f1-98cc-d6736328b7e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ac93704e4a34c95babf0f9ff5e76a37', 'request_id': 'req-17757dff-2115-45f1-98cc-d6736328b7e4'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:43.438313 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-17757dff-2115-45f1-98cc-d6736328b7e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ac93704e4a34c95babf0f9ff5e76a37', 'request_id': 'req-17757dff-2115-45f1-98cc-d6736328b7e4'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:43.543521 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-17757dff-2115-45f1-98cc-d6736328b7e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ac93704e4a34c95babf0f9ff5e76a37', 'request_id': 'req-17757dff-2115-45f1-98cc-d6736328b7e4'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:43.555509 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-17757dff-2115-45f1-98cc-d6736328b7e4 admin admin] The token cache is being invalidate because project 7ac93704e4a34c95babf0f9ff5e76a37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:14:43.555893 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-17757dff-2115-45f1-98cc-d6736328b7e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:43.558814 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-17757dff-2115-45f1-98cc-d6736328b7e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:43.560369 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1125/2237] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:14:43 2022] DELETE /identity/v3/projects/7ac93704e4a34c95babf0f9ff5e76a37 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:14:43.564575 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a68f669-311c-4e7d-aa1e-9cef53b1271d service cinder] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:43.564883 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7f997d-3b4d-4db0-8835-5d5ca5b61179 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:43.565298 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7f997d-3b4d-4db0-8835-5d5ca5b61179 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:43.565573 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7f997d-3b4d-4db0-8835-5d5ca5b61179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:43.573691 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a68f669-311c-4e7d-aa1e-9cef53b1271d service cinder] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:43.585728 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-8c7f997d-3b4d-4db0-8835-5d5ca5b61179 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:14:43.587080 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-8c7f997d-3b4d-4db0-8835-5d5ca5b61179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:43.620874 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:43.625663 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0495494e-54e5-42c0-a7d0-0ee990d217b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:43.626279 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0495494e-54e5-42c0-a7d0-0ee990d217b0 service cinder] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:43.626564 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0495494e-54e5-42c0-a7d0-0ee990d217b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:43.626841 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0495494e-54e5-42c0-a7d0-0ee990d217b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:43.628847 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-0495494e-54e5-42c0-a7d0-0ee990d217b0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:43.649017 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1126/2238] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:14:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:14:43.656433 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b15c62-1a37-47b8-ba4a-717d0cdd940e None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:43.656809 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b15c62-1a37-47b8-ba4a-717d0cdd940e None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:43.657153 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b15c62-1a37-47b8-ba4a-717d0cdd940e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:43.677174 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-93b15c62-1a37-47b8-ba4a-717d0cdd940e None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:14:43.678389 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-93b15c62-1a37-47b8-ba4a-717d0cdd940e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:43.690396 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1127/2239] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:14:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 17:14:43.691004 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b15c62-1a37-47b8-ba4a-717d0cdd940e None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:43.691765 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0495494e-54e5-42c0-a7d0-0ee990d217b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:43.693321 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0495494e-54e5-42c0-a7d0-0ee990d217b0 service cinder] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:43.701282 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b15c62-1a37-47b8-ba4a-717d0cdd940e None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:43.703741 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1113/2240] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:14:43 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:14:43.707213 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af32bb37-d768-4f2e-bdda-8d490ce47a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:43.707896 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af32bb37-d768-4f2e-bdda-8d490ce47a9c admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:43.708232 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af32bb37-d768-4f2e-bdda-8d490ce47a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:43.709245 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af32bb37-d768-4f2e-bdda-8d490ce47a9c admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:43.710151 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af32bb37-d768-4f2e-bdda-8d490ce47a9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:14:43.710515 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af32bb37-d768-4f2e-bdda-8d490ce47a9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:43.712266 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af32bb37-d768-4f2e-bdda-8d490ce47a9c admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:43.726796 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1128/2241] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:14:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 17:14:43.730849 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0495494e-54e5-42c0-a7d0-0ee990d217b0 service cinder] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:43.747229 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0495494e-54e5-42c0-a7d0-0ee990d217b0 service cinder] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:43.761799 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4603106e-90a5-47a2-bab8-efb9ad3bc71f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:43.767729 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4603106e-90a5-47a2-bab8-efb9ad3bc71f admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:43.767729 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4603106e-90a5-47a2-bab8-efb9ad3bc71f admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:43.767729 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4603106e-90a5-47a2-bab8-efb9ad3bc71f admin admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:43.767729 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4603106e-90a5-47a2-bab8-efb9ad3bc71f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:43.768444 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4603106e-90a5-47a2-bab8-efb9ad3bc71f admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:43.860239 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1114/2242] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:14:43 2022] POST /identity/v3/projects => generated 385 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:14:43.864238 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af32bb37-d768-4f2e-bdda-8d490ce47a9c admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:43.874785 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af32bb37-d768-4f2e-bdda-8d490ce47a9c admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:43.951215 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660e2647-4e86-43e9-86c0-7e5b9e79f5f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:43.951835 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660e2647-4e86-43e9-86c0-7e5b9e79f5f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:43.952136 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660e2647-4e86-43e9-86c0-7e5b9e79f5f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:43.952412 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660e2647-4e86-43e9-86c0-7e5b9e79f5f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:43.953086 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660e2647-4e86-43e9-86c0-7e5b9e79f5f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:43.955046 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660e2647-4e86-43e9-86c0-7e5b9e79f5f3 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:44.046816 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1129/2243] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:14:43 2022] POST /identity/v3/users => generated 293 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:14:44.050982 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4603106e-90a5-47a2-bab8-efb9ad3bc71f admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:44.061400 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4603106e-90a5-47a2-bab8-efb9ad3bc71f admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:44.065977 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656b38f1-59f7-4c20-b995-fb8c89942ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:44.066594 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656b38f1-59f7-4c20-b995-fb8c89942ead admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:44.066864 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656b38f1-59f7-4c20-b995-fb8c89942ead admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:44.067170 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656b38f1-59f7-4c20-b995-fb8c89942ead admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:44.068357 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656b38f1-59f7-4c20-b995-fb8c89942ead admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:44.070005 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656b38f1-59f7-4c20-b995-fb8c89942ead admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:44.085218 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1115/2244] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:14:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:14:44.086384 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660e2647-4e86-43e9-86c0-7e5b9e79f5f3 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:44.091558 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:44.096922 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660e2647-4e86-43e9-86c0-7e5b9e79f5f3 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:44.101842 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af3b0e4-a38e-4e17-ae90-1e5bc24bac6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:44.102447 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af3b0e4-a38e-4e17-ae90-1e5bc24bac6a admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:44.102823 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af3b0e4-a38e-4e17-ae90-1e5bc24bac6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:44.103038 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af3b0e4-a38e-4e17-ae90-1e5bc24bac6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:44.103675 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af3b0e4-a38e-4e17-ae90-1e5bc24bac6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:44.107509 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af3b0e4-a38e-4e17-ae90-1e5bc24bac6a admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:44.117357 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1130/2245] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:14:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:14:44.121470 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656b38f1-59f7-4c20-b995-fb8c89942ead admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:44.133213 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656b38f1-59f7-4c20-b995-fb8c89942ead admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:44.138109 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b33bb-a54f-49e1-9a4a-4785121379be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:44.138858 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b33bb-a54f-49e1-9a4a-4785121379be admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:44.139179 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b33bb-a54f-49e1-9a4a-4785121379be admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:44.139461 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b33bb-a54f-49e1-9a4a-4785121379be admin admin] PATH_INFO: `/v3/projects/2785773bcd4f47869fd9a9846ae174f4/users/b823488673d1491ebd533e259842a580/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:44.160479 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5b33bb-a54f-49e1-9a4a-4785121379be admin admin] RBAC: Authorizing `identity:create_grant(project_id=2785773bcd4f47869fd9a9846ae174f4, user_id=b823488673d1491ebd533e259842a580, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:44.164064 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5b33bb-a54f-49e1-9a4a-4785121379be admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:44.238383 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1116/2246] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:14:44 2022] PUT /identity/v3/projects/2785773bcd4f47869fd9a9846ae174f4/users/b823488673d1491ebd533e259842a580/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:14:44.250734 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4abaa8-4dae-48ea-aeeb-f78625dd65ad None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:44.251066 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4abaa8-4dae-48ea-aeeb-f78625dd65ad None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:44.251367 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4abaa8-4dae-48ea-aeeb-f78625dd65ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:44.281276 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-de4abaa8-4dae-48ea-aeeb-f78625dd65ad None None] MFA Rules not processed for user `b823488673d1491ebd533e259842a580`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:14:44.282609 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-de4abaa8-4dae-48ea-aeeb-f78625dd65ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:44.346805 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1131/2247] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:14:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:14:44.351810 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b33bb-a54f-49e1-9a4a-4785121379be admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:44.417520 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b33bb-a54f-49e1-9a4a-4785121379be admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:44.423167 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b6429d-029a-40db-9d0c-9ab26a67bf72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:44.423839 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b6429d-029a-40db-9d0c-9ab26a67bf72 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:44.424126 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b6429d-029a-40db-9d0c-9ab26a67bf72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:44.424404 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b6429d-029a-40db-9d0c-9ab26a67bf72 admin admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:44.425091 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b6429d-029a-40db-9d0c-9ab26a67bf72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:44.427183 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b6429d-029a-40db-9d0c-9ab26a67bf72 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:44.547518 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1117/2248] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:14:44 2022] POST /identity/v3/projects => generated 387 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:14:44.551844 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4abaa8-4dae-48ea-aeeb-f78625dd65ad None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:44.562635 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4abaa8-4dae-48ea-aeeb-f78625dd65ad None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:44.618679 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b523fd4-5e07-4133-9c90-fc2d12e72485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:44.619457 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b523fd4-5e07-4133-9c90-fc2d12e72485 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:44.619737 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b523fd4-5e07-4133-9c90-fc2d12e72485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:44.620009 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b523fd4-5e07-4133-9c90-fc2d12e72485 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:44.620622 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b523fd4-5e07-4133-9c90-fc2d12e72485 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:44.622522 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b523fd4-5e07-4133-9c90-fc2d12e72485 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:44.719839 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1132/2249] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:14:44 2022] POST /identity/v3/users => generated 300 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:14:44.723824 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b6429d-029a-40db-9d0c-9ab26a67bf72 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:44.734889 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b6429d-029a-40db-9d0c-9ab26a67bf72 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:44.739686 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7d21fa-dffe-48c8-9772-356a38e9c02e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:44.740566 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d21fa-dffe-48c8-9772-356a38e9c02e admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:44.740850 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d21fa-dffe-48c8-9772-356a38e9c02e admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:44.741169 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d21fa-dffe-48c8-9772-356a38e9c02e admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:44.746759 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7d21fa-dffe-48c8-9772-356a38e9c02e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:44.746759 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7d21fa-dffe-48c8-9772-356a38e9c02e admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:44.753638 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1118/2250] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:14:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:14:44.758331 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b523fd4-5e07-4133-9c90-fc2d12e72485 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:44.768683 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b523fd4-5e07-4133-9c90-fc2d12e72485 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:44.774135 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb64798d-2006-4f57-9d0b-57cd3196127a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:44.774823 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb64798d-2006-4f57-9d0b-57cd3196127a admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:44.775177 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb64798d-2006-4f57-9d0b-57cd3196127a admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:44.775460 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb64798d-2006-4f57-9d0b-57cd3196127a admin admin] PATH_INFO: `/v3/projects/494f00bbdcbd4768a03c4f45a66d7a1e/users/996bdb958d5c4e178c04c39099c6c5d4/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:44.790888 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb64798d-2006-4f57-9d0b-57cd3196127a admin admin] RBAC: Authorizing `identity:create_grant(project_id=494f00bbdcbd4768a03c4f45a66d7a1e, user_id=996bdb958d5c4e178c04c39099c6c5d4, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:44.793692 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb64798d-2006-4f57-9d0b-57cd3196127a admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:44.862561 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1133/2251] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:14:44 2022] PUT /identity/v3/projects/494f00bbdcbd4768a03c4f45a66d7a1e/users/996bdb958d5c4e178c04c39099c6c5d4/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:14:44.866758 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7d21fa-dffe-48c8-9772-356a38e9c02e admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:44.876932 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7d21fa-dffe-48c8-9772-356a38e9c02e admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:44.893922 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:44.933659 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afae9af7-7107-4a66-a864-194f009a102e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:44.934348 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae9af7-7107-4a66-a864-194f009a102e admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:44.934709 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae9af7-7107-4a66-a864-194f009a102e admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:44.935015 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae9af7-7107-4a66-a864-194f009a102e admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:44.935710 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afae9af7-7107-4a66-a864-194f009a102e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:44.937372 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afae9af7-7107-4a66-a864-194f009a102e admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:44.948110 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1119/2252] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:14:44 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:14:44.952269 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb64798d-2006-4f57-9d0b-57cd3196127a admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:44.963381 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb64798d-2006-4f57-9d0b-57cd3196127a admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:44.968256 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e5950f-9ce2-4610-a181-9b546b2c55e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:44.968915 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e5950f-9ce2-4610-a181-9b546b2c55e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:44.969194 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e5950f-9ce2-4610-a181-9b546b2c55e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:44.969466 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e5950f-9ce2-4610-a181-9b546b2c55e3 admin admin] PATH_INFO: `/v3/projects/494f00bbdcbd4768a03c4f45a66d7a1e/users/996bdb958d5c4e178c04c39099c6c5d4/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:44.971426 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e5950f-9ce2-4610-a181-9b546b2c55e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=494f00bbdcbd4768a03c4f45a66d7a1e, user_id=996bdb958d5c4e178c04c39099c6c5d4, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:44.974219 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e5950f-9ce2-4610-a181-9b546b2c55e3 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:44.987242 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1134/2253] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:14:44 2022] PUT /identity/v3/projects/494f00bbdcbd4768a03c4f45a66d7a1e/users/996bdb958d5c4e178c04c39099c6c5d4/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:14:44.998296 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956697d-61c1-4f81-be13-326dd6d2cd56 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:44.998588 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956697d-61c1-4f81-be13-326dd6d2cd56 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:44.998867 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956697d-61c1-4f81-be13-326dd6d2cd56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:45.034084 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-9956697d-61c1-4f81-be13-326dd6d2cd56 None None] MFA Rules not processed for user `996bdb958d5c4e178c04c39099c6c5d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:14:45.035346 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-9956697d-61c1-4f81-be13-326dd6d2cd56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:45.106443 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1120/2254] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:14:44 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:14:45.118467 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e5950f-9ce2-4610-a181-9b546b2c55e3 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:45.120254 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-d3e5950f-9ce2-4610-a181-9b546b2c55e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:45.174688 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:45.210377 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e5950f-9ce2-4610-a181-9b546b2c55e3 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:45.214168 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39344f3b-cf51-4a99-840f-21b967eee7d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:45.214802 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39344f3b-cf51-4a99-840f-21b967eee7d3 service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:45.219017 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39344f3b-cf51-4a99-840f-21b967eee7d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:45.219387 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39344f3b-cf51-4a99-840f-21b967eee7d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:45.232680 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39344f3b-cf51-4a99-840f-21b967eee7d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:45.234267 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39344f3b-cf51-4a99-840f-21b967eee7d3 service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:45.247973 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1135/2255] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:14:45 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:14:45.277899 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9956697d-61c1-4f81-be13-326dd6d2cd56 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:45.349522 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9956697d-61c1-4f81-be13-326dd6d2cd56 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:45.354067 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732e4ebc-4340-4073-b235-e0034d421305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:45.354696 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732e4ebc-4340-4073-b235-e0034d421305 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:45.356127 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732e4ebc-4340-4073-b235-e0034d421305 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:45.356127 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732e4ebc-4340-4073-b235-e0034d421305 admin admin] PATH_INFO: `/v3/users/b823488673d1491ebd533e259842a580` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:45.356507 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732e4ebc-4340-4073-b235-e0034d421305 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b823488673d1491ebd533e259842a580)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:45.358453 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732e4ebc-4340-4073-b235-e0034d421305 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:45.629033 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:45.633666 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-732e4ebc-4340-4073-b235-e0034d421305 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b823488673d1491ebd533e259842a580'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:45.654647 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-732e4ebc-4340-4073-b235-e0034d421305 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b823488673d1491ebd533e259842a580'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:45.655672 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-732e4ebc-4340-4073-b235-e0034d421305 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b823488673d1491ebd533e259842a580'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:45.750037 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-732e4ebc-4340-4073-b235-e0034d421305 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b823488673d1491ebd533e259842a580'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:45.751683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-732e4ebc-4340-4073-b235-e0034d421305 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b823488673d1491ebd533e259842a580'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:45.770447 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1121/2256] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:14:45 2022] DELETE /identity/v3/users/b823488673d1491ebd533e259842a580 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:14:45.774889 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eafb637-f987-49d3-a3f0-4be5e38b4362 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:45.775387 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eafb637-f987-49d3-a3f0-4be5e38b4362 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:45.775511 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eafb637-f987-49d3-a3f0-4be5e38b4362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:45.804852 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-6eafb637-f987-49d3-a3f0-4be5e38b4362 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:14:45.806776 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-6eafb637-f987-49d3-a3f0-4be5e38b4362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:45.881445 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1136/2257] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:14:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:14:45.892066 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732e4ebc-4340-4073-b235-e0034d421305 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:45.893347 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-732e4ebc-4340-4073-b235-e0034d421305 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:45.952705 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732e4ebc-4340-4073-b235-e0034d421305 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:45.956520 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0258585f-8a41-4074-941c-265cfc6b9df0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:45.957122 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0258585f-8a41-4074-941c-265cfc6b9df0 service neutron] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:45.957471 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0258585f-8a41-4074-941c-265cfc6b9df0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:45.957812 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0258585f-8a41-4074-941c-265cfc6b9df0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:45.967283 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0258585f-8a41-4074-941c-265cfc6b9df0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:45.968751 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0258585f-8a41-4074-941c-265cfc6b9df0 service neutron] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:45.980217 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1122/2258] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:14:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:14:46.511166 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eafb637-f987-49d3-a3f0-4be5e38b4362 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:46.512665 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-6eafb637-f987-49d3-a3f0-4be5e38b4362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:46.526241 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eafb637-f987-49d3-a3f0-4be5e38b4362 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:46.532484 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed51af6-862f-4639-b272-a7d5eb43a071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:46.533783 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed51af6-862f-4639-b272-a7d5eb43a071 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:46.534198 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed51af6-862f-4639-b272-a7d5eb43a071 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:46.534827 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed51af6-862f-4639-b272-a7d5eb43a071 admin admin] PATH_INFO: `/v3/projects/2785773bcd4f47869fd9a9846ae174f4` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:46.536884 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed51af6-862f-4639-b272-a7d5eb43a071 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2785773bcd4f47869fd9a9846ae174f4)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:46.539727 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed51af6-862f-4639-b272-a7d5eb43a071 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:46.735584 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-8ed51af6-862f-4639-b272-a7d5eb43a071 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2785773bcd4f47869fd9a9846ae174f4', 'request_id': 'req-8ed51af6-862f-4639-b272-a7d5eb43a071'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:46.746022 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-8ed51af6-862f-4639-b272-a7d5eb43a071 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2785773bcd4f47869fd9a9846ae174f4', 'request_id': 'req-8ed51af6-862f-4639-b272-a7d5eb43a071'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:46.849651 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-8ed51af6-862f-4639-b272-a7d5eb43a071 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2785773bcd4f47869fd9a9846ae174f4', 'request_id': 'req-8ed51af6-862f-4639-b272-a7d5eb43a071'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:46.864260 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-8ed51af6-862f-4639-b272-a7d5eb43a071 admin admin] The token cache is being invalidate because project 2785773bcd4f47869fd9a9846ae174f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:14:46.869217 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-8ed51af6-862f-4639-b272-a7d5eb43a071 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:46.869217 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-8ed51af6-862f-4639-b272-a7d5eb43a071 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:46.869217 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1137/2259] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:14:46 2022] DELETE /identity/v3/projects/2785773bcd4f47869fd9a9846ae174f4 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:14:46.872117 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0258585f-8a41-4074-941c-265cfc6b9df0 service neutron] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:46.882802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0258585f-8a41-4074-941c-265cfc6b9df0 service neutron] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:46.945316 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26970a6d-8790-4a9a-8438-739f029dfca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:46.945958 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26970a6d-8790-4a9a-8438-739f029dfca3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:46.946239 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26970a6d-8790-4a9a-8438-739f029dfca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:46.946512 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26970a6d-8790-4a9a-8438-739f029dfca3 admin admin] PATH_INFO: `/v3/users/996bdb958d5c4e178c04c39099c6c5d4` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:46.947672 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26970a6d-8790-4a9a-8438-739f029dfca3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=996bdb958d5c4e178c04c39099c6c5d4)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:46.949582 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26970a6d-8790-4a9a-8438-739f029dfca3 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:47.137325 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-26970a6d-8790-4a9a-8438-739f029dfca3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '996bdb958d5c4e178c04c39099c6c5d4'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:47.152238 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-26970a6d-8790-4a9a-8438-739f029dfca3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '996bdb958d5c4e178c04c39099c6c5d4'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:47.153355 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-26970a6d-8790-4a9a-8438-739f029dfca3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '996bdb958d5c4e178c04c39099c6c5d4'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:47.155903 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:47.248704 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-26970a6d-8790-4a9a-8438-739f029dfca3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '996bdb958d5c4e178c04c39099c6c5d4'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:47.249296 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-26970a6d-8790-4a9a-8438-739f029dfca3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '996bdb958d5c4e178c04c39099c6c5d4'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:47.262988 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1123/2260] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:14:46 2022] DELETE /identity/v3/users/996bdb958d5c4e178c04c39099c6c5d4 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:14:47.651604 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed51af6-862f-4639-b272-a7d5eb43a071 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:47.653514 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-8ed51af6-862f-4639-b272-a7d5eb43a071 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:47.675367 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed51af6-862f-4639-b272-a7d5eb43a071 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:47.745588 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c40672a-3f42-4b01-adf3-8d62f49caba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:47.746363 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c40672a-3f42-4b01-adf3-8d62f49caba6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:47.746737 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c40672a-3f42-4b01-adf3-8d62f49caba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:47.747118 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c40672a-3f42-4b01-adf3-8d62f49caba6 admin admin] PATH_INFO: `/v3/projects/494f00bbdcbd4768a03c4f45a66d7a1e` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:47.748286 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c40672a-3f42-4b01-adf3-8d62f49caba6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=494f00bbdcbd4768a03c4f45a66d7a1e)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:47.750379 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c40672a-3f42-4b01-adf3-8d62f49caba6 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:47.751954 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:47.870229 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-3c40672a-3f42-4b01-adf3-8d62f49caba6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '494f00bbdcbd4768a03c4f45a66d7a1e', 'request_id': 'req-3c40672a-3f42-4b01-adf3-8d62f49caba6'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:47.879388 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-3c40672a-3f42-4b01-adf3-8d62f49caba6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '494f00bbdcbd4768a03c4f45a66d7a1e', 'request_id': 'req-3c40672a-3f42-4b01-adf3-8d62f49caba6'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:47.954539 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-3c40672a-3f42-4b01-adf3-8d62f49caba6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '494f00bbdcbd4768a03c4f45a66d7a1e', 'request_id': 'req-3c40672a-3f42-4b01-adf3-8d62f49caba6'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:47.965998 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-3c40672a-3f42-4b01-adf3-8d62f49caba6 admin admin] The token cache is being invalidate because project 494f00bbdcbd4768a03c4f45a66d7a1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:14:47.966291 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-3c40672a-3f42-4b01-adf3-8d62f49caba6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:47.966980 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-3c40672a-3f42-4b01-adf3-8d62f49caba6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:14:47.968480 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1138/2261] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:14:47 2022] DELETE /identity/v3/projects/494f00bbdcbd4768a03c4f45a66d7a1e => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:14:47.974210 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9946937f-8a38-4d23-8b4d-03c8579a7e40 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:47.974998 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9946937f-8a38-4d23-8b4d-03c8579a7e40 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:47.974998 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9946937f-8a38-4d23-8b4d-03c8579a7e40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:47.995030 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-9946937f-8a38-4d23-8b4d-03c8579a7e40 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:14:47.996266 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-9946937f-8a38-4d23-8b4d-03c8579a7e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:48.055042 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1124/2262] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:14:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:14:48.062796 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cdff2c-771b-43d2-b2d2-58c3a0698909 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:48.063123 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cdff2c-771b-43d2-b2d2-58c3a0698909 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:48.063405 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cdff2c-771b-43d2-b2d2-58c3a0698909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:48.083696 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-e8cdff2c-771b-43d2-b2d2-58c3a0698909 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:14:48.084901 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-e8cdff2c-771b-43d2-b2d2-58c3a0698909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:48.093073 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1139/2263] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:14:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:14:48.097320 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9946937f-8a38-4d23-8b4d-03c8579a7e40 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:48.107524 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9946937f-8a38-4d23-8b4d-03c8579a7e40 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:48.112133 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acc64d3-abd1-4cba-8018-af2c68caca1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:48.112731 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc64d3-abd1-4cba-8018-af2c68caca1c admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:48.113065 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc64d3-abd1-4cba-8018-af2c68caca1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:48.113338 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc64d3-abd1-4cba-8018-af2c68caca1c admin admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:48.113983 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acc64d3-abd1-4cba-8018-af2c68caca1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:14:48.114352 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acc64d3-abd1-4cba-8018-af2c68caca1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:48.116102 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acc64d3-abd1-4cba-8018-af2c68caca1c admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:48.128285 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1125/2264] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:14:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 17:14:48.132840 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cdff2c-771b-43d2-b2d2-58c3a0698909 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:48.142492 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cdff2c-771b-43d2-b2d2-58c3a0698909 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:48.148161 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550b2a84-5af0-4d73-b6f0-0516c81b3892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:48.148775 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550b2a84-5af0-4d73-b6f0-0516c81b3892 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:48.149057 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550b2a84-5af0-4d73-b6f0-0516c81b3892 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:48.149332 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550b2a84-5af0-4d73-b6f0-0516c81b3892 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:48.150000 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550b2a84-5af0-4d73-b6f0-0516c81b3892 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:48.151945 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550b2a84-5af0-4d73-b6f0-0516c81b3892 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:48.272670 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1140/2265] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:14:48 2022] POST /identity/v3/projects => generated 387 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:14:48.277125 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acc64d3-abd1-4cba-8018-af2c68caca1c admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:48.287744 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acc64d3-abd1-4cba-8018-af2c68caca1c admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:48.343743 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f460e6de-dd44-474d-9616-e308503d8eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:48.344255 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f460e6de-dd44-474d-9616-e308503d8eac admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:48.344536 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f460e6de-dd44-474d-9616-e308503d8eac admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:48.344809 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f460e6de-dd44-474d-9616-e308503d8eac admin admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:48.345463 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f460e6de-dd44-474d-9616-e308503d8eac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:48.347418 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f460e6de-dd44-474d-9616-e308503d8eac admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:48.445824 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1126/2266] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:14:48 2022] POST /identity/v3/users => generated 294 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:14:48.447372 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550b2a84-5af0-4d73-b6f0-0516c81b3892 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:48.457884 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550b2a84-5af0-4d73-b6f0-0516c81b3892 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:48.466330 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbef98e-4b92-48f0-8032-f3d89cc5869f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:48.467007 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbef98e-4b92-48f0-8032-f3d89cc5869f admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:48.467272 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbef98e-4b92-48f0-8032-f3d89cc5869f admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:48.472226 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbef98e-4b92-48f0-8032-f3d89cc5869f admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:48.472226 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbef98e-4b92-48f0-8032-f3d89cc5869f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:48.472226 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbef98e-4b92-48f0-8032-f3d89cc5869f admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:48.481125 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1141/2267] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:14:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:14:48.484793 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f460e6de-dd44-474d-9616-e308503d8eac admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:48.495075 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f460e6de-dd44-474d-9616-e308503d8eac admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:48.500339 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0b0195-afbb-4784-9d38-5c42d226eae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:48.512535 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b0195-afbb-4784-9d38-5c42d226eae0 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:48.512535 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b0195-afbb-4784-9d38-5c42d226eae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:48.512535 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b0195-afbb-4784-9d38-5c42d226eae0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:48.512535 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0b0195-afbb-4784-9d38-5c42d226eae0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:48.512535 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0b0195-afbb-4784-9d38-5c42d226eae0 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:48.515863 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1127/2268] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:14:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:14:48.520132 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbef98e-4b92-48f0-8032-f3d89cc5869f admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:48.530835 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbef98e-4b92-48f0-8032-f3d89cc5869f admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:48.535544 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd80d65b-cc6a-4699-85aa-292c6e32e898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:48.536812 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd80d65b-cc6a-4699-85aa-292c6e32e898 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:48.537170 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd80d65b-cc6a-4699-85aa-292c6e32e898 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:48.537467 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd80d65b-cc6a-4699-85aa-292c6e32e898 admin admin] PATH_INFO: `/v3/projects/67b228134d574af0a6c228c145c8737f/users/d38b8e7cc88841fb8ef93dd4c5c81417/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:48.557254 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd80d65b-cc6a-4699-85aa-292c6e32e898 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67b228134d574af0a6c228c145c8737f, user_id=d38b8e7cc88841fb8ef93dd4c5c81417, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:48.560729 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd80d65b-cc6a-4699-85aa-292c6e32e898 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:48.642597 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1142/2269] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:14:48 2022] PUT /identity/v3/projects/67b228134d574af0a6c228c145c8737f/users/d38b8e7cc88841fb8ef93dd4c5c81417/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:14:48.654185 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0e3f2d-2876-48a2-a099-77062f56800a None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:48.654482 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0e3f2d-2876-48a2-a099-77062f56800a None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:48.654748 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0e3f2d-2876-48a2-a099-77062f56800a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:48.683131 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-3e0e3f2d-2876-48a2-a099-77062f56800a None None] MFA Rules not processed for user `d38b8e7cc88841fb8ef93dd4c5c81417`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:14:48.684352 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-3e0e3f2d-2876-48a2-a099-77062f56800a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:48.748470 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1128/2270] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:14:48 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:14:48.753101 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd80d65b-cc6a-4699-85aa-292c6e32e898 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:48.814997 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd80d65b-cc6a-4699-85aa-292c6e32e898 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:48.820016 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19bd787-cd61-4d28-b88c-f44b39421045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:48.820626 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19bd787-cd61-4d28-b88c-f44b39421045 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:48.820910 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19bd787-cd61-4d28-b88c-f44b39421045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:48.821186 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19bd787-cd61-4d28-b88c-f44b39421045 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:48.821822 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19bd787-cd61-4d28-b88c-f44b39421045 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:48.823823 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19bd787-cd61-4d28-b88c-f44b39421045 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:48.923926 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1143/2271] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:14:48 2022] POST /identity/v3/projects => generated 387 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:14:48.928609 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0e3f2d-2876-48a2-a099-77062f56800a None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:48.939183 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0e3f2d-2876-48a2-a099-77062f56800a None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:48.999721 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f175b3a-9a0f-43bb-95ca-d4769c23795c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:49.000444 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f175b3a-9a0f-43bb-95ca-d4769c23795c admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:49.000803 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f175b3a-9a0f-43bb-95ca-d4769c23795c admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:49.001188 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f175b3a-9a0f-43bb-95ca-d4769c23795c admin admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:49.002119 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f175b3a-9a0f-43bb-95ca-d4769c23795c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:49.004269 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f175b3a-9a0f-43bb-95ca-d4769c23795c admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:49.087041 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1129/2272] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:14:48 2022] POST /identity/v3/users => generated 294 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:14:49.092025 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19bd787-cd61-4d28-b88c-f44b39421045 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:49.103311 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19bd787-cd61-4d28-b88c-f44b39421045 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:49.116387 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046ed1e4-361a-4725-a14a-ebf74fce24bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:49.117028 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ed1e4-361a-4725-a14a-ebf74fce24bb admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:49.117314 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ed1e4-361a-4725-a14a-ebf74fce24bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:49.117592 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ed1e4-361a-4725-a14a-ebf74fce24bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:49.118237 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046ed1e4-361a-4725-a14a-ebf74fce24bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:49.119890 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046ed1e4-361a-4725-a14a-ebf74fce24bb admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:49.134129 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1144/2273] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:14:49 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:14:49.138135 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f175b3a-9a0f-43bb-95ca-d4769c23795c admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:49.149981 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f175b3a-9a0f-43bb-95ca-d4769c23795c admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:49.155636 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b6f601-2bf4-4dcf-845e-79051fb2cc74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:49.156287 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b6f601-2bf4-4dcf-845e-79051fb2cc74 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:49.157209 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b6f601-2bf4-4dcf-845e-79051fb2cc74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:49.157596 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b6f601-2bf4-4dcf-845e-79051fb2cc74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:49.158124 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b6f601-2bf4-4dcf-845e-79051fb2cc74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:49.159838 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b6f601-2bf4-4dcf-845e-79051fb2cc74 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:49.171615 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1130/2274] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:14:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:14:49.175776 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046ed1e4-361a-4725-a14a-ebf74fce24bb admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:49.186204 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046ed1e4-361a-4725-a14a-ebf74fce24bb admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:49.190980 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fafce1-01db-4532-8d52-6a94f6d5f20e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:49.193459 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fafce1-01db-4532-8d52-6a94f6d5f20e admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:49.193459 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fafce1-01db-4532-8d52-6a94f6d5f20e admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:49.193459 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fafce1-01db-4532-8d52-6a94f6d5f20e admin admin] PATH_INFO: `/v3/projects/aef6084e9e8c4d1dafe4e5b1dda9c276/users/93ab1d4c30f94403b762ebc2a60d307f/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:49.206710 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fafce1-01db-4532-8d52-6a94f6d5f20e admin admin] RBAC: Authorizing `identity:create_grant(project_id=aef6084e9e8c4d1dafe4e5b1dda9c276, user_id=93ab1d4c30f94403b762ebc2a60d307f, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:49.209968 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fafce1-01db-4532-8d52-6a94f6d5f20e admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:49.318809 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1145/2275] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:14:49 2022] PUT /identity/v3/projects/aef6084e9e8c4d1dafe4e5b1dda9c276/users/93ab1d4c30f94403b762ebc2a60d307f/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:14:49.329026 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6068f54e-c4bf-4998-ba23-7fcd188d6d57 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:49.329357 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6068f54e-c4bf-4998-ba23-7fcd188d6d57 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:49.329635 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6068f54e-c4bf-4998-ba23-7fcd188d6d57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:49.358760 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-6068f54e-c4bf-4998-ba23-7fcd188d6d57 None None] MFA Rules not processed for user `93ab1d4c30f94403b762ebc2a60d307f`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:14:49.360158 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-6068f54e-c4bf-4998-ba23-7fcd188d6d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:49.426252 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1131/2276] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:14:49 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:14:49.430860 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fafce1-01db-4532-8d52-6a94f6d5f20e admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:49.494859 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fafce1-01db-4532-8d52-6a94f6d5f20e admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:49.499776 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8576fb8f-7211-4914-ac9b-62333738c418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:49.500378 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8576fb8f-7211-4914-ac9b-62333738c418 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:49.500655 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8576fb8f-7211-4914-ac9b-62333738c418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:49.505544 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8576fb8f-7211-4914-ac9b-62333738c418 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:49.505544 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8576fb8f-7211-4914-ac9b-62333738c418 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:49.505544 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8576fb8f-7211-4914-ac9b-62333738c418 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:49.545851 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:49.575174 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1146/2277] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:14:49 2022] POST /identity/v3/projects => generated 385 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:14:49.579169 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6068f54e-c4bf-4998-ba23-7fcd188d6d57 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:49.590825 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6068f54e-c4bf-4998-ba23-7fcd188d6d57 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:49.647962 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01807511-bc2b-4496-95e3-d4da3b690a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:49.648601 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01807511-bc2b-4496-95e3-d4da3b690a7f admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:49.648884 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01807511-bc2b-4496-95e3-d4da3b690a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:49.649158 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01807511-bc2b-4496-95e3-d4da3b690a7f admin admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:49.649906 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01807511-bc2b-4496-95e3-d4da3b690a7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:49.651865 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01807511-bc2b-4496-95e3-d4da3b690a7f admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:49.688781 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:49.752496 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1132/2278] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:14:49 2022] POST /identity/v3/users => generated 299 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:14:49.757037 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8576fb8f-7211-4914-ac9b-62333738c418 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:49.768094 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8576fb8f-7211-4914-ac9b-62333738c418 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:49.774624 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a6eab5-abd7-409e-8975-1d1067f204ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:49.775353 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a6eab5-abd7-409e-8975-1d1067f204ff admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:49.775697 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a6eab5-abd7-409e-8975-1d1067f204ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:49.776196 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a6eab5-abd7-409e-8975-1d1067f204ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:49.777156 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a6eab5-abd7-409e-8975-1d1067f204ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:49.778602 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a6eab5-abd7-409e-8975-1d1067f204ff admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:49.795612 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1147/2279] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:14:49 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 17:14:49.799726 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01807511-bc2b-4496-95e3-d4da3b690a7f admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:49.810809 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01807511-bc2b-4496-95e3-d4da3b690a7f admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:49.815441 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d257cdfa-de18-4bb2-bd47-f65890fdaa41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:49.816098 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257cdfa-de18-4bb2-bd47-f65890fdaa41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:49.816380 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257cdfa-de18-4bb2-bd47-f65890fdaa41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:49.816651 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257cdfa-de18-4bb2-bd47-f65890fdaa41 admin admin] PATH_INFO: `/v3/projects/773023aa750f4b389c0cddace43a0962/users/269417db723f4556bd568ca762115d21/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:49.838840 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d257cdfa-de18-4bb2-bd47-f65890fdaa41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=773023aa750f4b389c0cddace43a0962, user_id=269417db723f4556bd568ca762115d21, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:49.841751 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d257cdfa-de18-4bb2-bd47-f65890fdaa41 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:49.922012 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1133/2280] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:14:49 2022] PUT /identity/v3/projects/773023aa750f4b389c0cddace43a0962/users/269417db723f4556bd568ca762115d21/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:14:49.925929 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a6eab5-abd7-409e-8975-1d1067f204ff admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:49.936329 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a6eab5-abd7-409e-8975-1d1067f204ff admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:49.993882 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf3dbdd-e2f0-4369-8134-b44c00935012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:49.994540 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf3dbdd-e2f0-4369-8134-b44c00935012 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:49.994819 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf3dbdd-e2f0-4369-8134-b44c00935012 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:49.995120 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf3dbdd-e2f0-4369-8134-b44c00935012 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:49.995764 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf3dbdd-e2f0-4369-8134-b44c00935012 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:49.997362 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf3dbdd-e2f0-4369-8134-b44c00935012 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:50.008511 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1148/2281] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:14:49 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 17:14:50.012819 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d257cdfa-de18-4bb2-bd47-f65890fdaa41 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:50.023386 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d257cdfa-de18-4bb2-bd47-f65890fdaa41 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:50.028065 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4a1881-0383-46bb-a9db-ff9a3ea815c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:50.028809 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4a1881-0383-46bb-a9db-ff9a3ea815c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:50.029161 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4a1881-0383-46bb-a9db-ff9a3ea815c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:50.029500 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4a1881-0383-46bb-a9db-ff9a3ea815c3 admin admin] PATH_INFO: `/v3/projects/773023aa750f4b389c0cddace43a0962/users/269417db723f4556bd568ca762115d21/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:50.031536 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4a1881-0383-46bb-a9db-ff9a3ea815c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=773023aa750f4b389c0cddace43a0962, user_id=269417db723f4556bd568ca762115d21, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:50.034241 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4a1881-0383-46bb-a9db-ff9a3ea815c3 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:50.045982 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1134/2282] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:14:50 2022] PUT /identity/v3/projects/773023aa750f4b389c0cddace43a0962/users/269417db723f4556bd568ca762115d21/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:14:50.057330 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb7c251-5d4c-4d59-9f43-1a27d39cb96a None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:50.058052 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb7c251-5d4c-4d59-9f43-1a27d39cb96a None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:50.058680 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb7c251-5d4c-4d59-9f43-1a27d39cb96a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:50.084573 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:50.096750 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-5fb7c251-5d4c-4d59-9f43-1a27d39cb96a None None] MFA Rules not processed for user `269417db723f4556bd568ca762115d21`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:14:50.098460 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-5fb7c251-5d4c-4d59-9f43-1a27d39cb96a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:50.172748 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1149/2283] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:14:50 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 17:14:50.183787 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4a1881-0383-46bb-a9db-ff9a3ea815c3 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:50.185614 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-4e4a1881-0383-46bb-a9db-ff9a3ea815c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:50.265196 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4a1881-0383-46bb-a9db-ff9a3ea815c3 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:50.265196 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18958c5-289d-4884-a1f4-449954e04537 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:50.265196 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18958c5-289d-4884-a1f4-449954e04537 service cinder] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:50.265196 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18958c5-289d-4884-a1f4-449954e04537 service cinder] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:50.265555 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18958c5-289d-4884-a1f4-449954e04537 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:50.269534 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18958c5-289d-4884-a1f4-449954e04537 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:50.271700 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18958c5-289d-4884-a1f4-449954e04537 service cinder] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:50.274727 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:14:50.281794 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1135/2284] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:14:50 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:14:51.447601 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb7c251-5d4c-4d59-9f43-1a27d39cb96a None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:51.461536 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb7c251-5d4c-4d59-9f43-1a27d39cb96a None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:51.465548 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b1324e-88a8-4eae-b6bc-3b73b370c681 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:51.466167 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b1324e-88a8-4eae-b6bc-3b73b370c681 service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:51.466592 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b1324e-88a8-4eae-b6bc-3b73b370c681 service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:51.466877 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b1324e-88a8-4eae-b6bc-3b73b370c681 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:51.528532 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b1324e-88a8-4eae-b6bc-3b73b370c681 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:51.530409 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b1324e-88a8-4eae-b6bc-3b73b370c681 service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:51.540864 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1150/2285] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:14:51 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:14:59.714709 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18958c5-289d-4884-a1f4-449954e04537 service cinder] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:14:59.716718 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-b18958c5-289d-4884-a1f4-449954e04537 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:14:59.727960 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18958c5-289d-4884-a1f4-449954e04537 service cinder] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:14:59.731631 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a6b354-8527-437f-8afb-97f5ef356cb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:14:59.732281 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a6b354-8527-437f-8afb-97f5ef356cb5 service cinder] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:14:59.732567 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a6b354-8527-437f-8afb-97f5ef356cb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:14:59.732838 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a6b354-8527-437f-8afb-97f5ef356cb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:14:59.810528 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a6b354-8527-437f-8afb-97f5ef356cb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:14:59.812665 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a6b354-8527-437f-8afb-97f5ef356cb5 service cinder] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:14:59.821563 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1136/2286] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:14:59 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:15:01.535791 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=41 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:15:02.096020 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b1324e-88a8-4eae-b6bc-3b73b370c681 service cinder] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:02.100292 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-47b1324e-88a8-4eae-b6bc-3b73b370c681 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:02.171394 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b1324e-88a8-4eae-b6bc-3b73b370c681 service cinder] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:02.176515 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d6651-2ebe-44b0-9f1b-fb898e631144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:02.177206 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d6651-2ebe-44b0-9f1b-fb898e631144 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:02.177488 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d6651-2ebe-44b0-9f1b-fb898e631144 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:02.181051 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d6651-2ebe-44b0-9f1b-fb898e631144 admin admin] PATH_INFO: `/v3/users/3c9baff1a79143199c3b0dc5afb66383` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:02.181051 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910d6651-2ebe-44b0-9f1b-fb898e631144 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c9baff1a79143199c3b0dc5afb66383)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:02.181051 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910d6651-2ebe-44b0-9f1b-fb898e631144 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:02.416836 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-910d6651-2ebe-44b0-9f1b-fb898e631144 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c9baff1a79143199c3b0dc5afb66383'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:02.525904 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-910d6651-2ebe-44b0-9f1b-fb898e631144 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c9baff1a79143199c3b0dc5afb66383'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:02.526804 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-910d6651-2ebe-44b0-9f1b-fb898e631144 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c9baff1a79143199c3b0dc5afb66383'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:02.527445 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-910d6651-2ebe-44b0-9f1b-fb898e631144 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c9baff1a79143199c3b0dc5afb66383'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:02.535284 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-910d6651-2ebe-44b0-9f1b-fb898e631144 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c9baff1a79143199c3b0dc5afb66383'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:02.553764 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1151/2287] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:15:02 2022] DELETE /identity/v3/users/3c9baff1a79143199c3b0dc5afb66383 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:15:02.558351 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fccac47-cd5d-406b-b53d-3fd9a9fcdf57 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:02.558786 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fccac47-cd5d-406b-b53d-3fd9a9fcdf57 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:02.558978 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fccac47-cd5d-406b-b53d-3fd9a9fcdf57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:02.585426 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-4fccac47-cd5d-406b-b53d-3fd9a9fcdf57 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:15:02.586630 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-4fccac47-cd5d-406b-b53d-3fd9a9fcdf57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:02.620486 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:15:02.656613 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1137/2288] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:15:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:15:02.672225 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d6651-2ebe-44b0-9f1b-fb898e631144 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:02.674222 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-910d6651-2ebe-44b0-9f1b-fb898e631144 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:02.743081 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d6651-2ebe-44b0-9f1b-fb898e631144 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:02.747114 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde474a9-e888-4f2d-9299-2d2a64ef5d62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:02.747475 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde474a9-e888-4f2d-9299-2d2a64ef5d62 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:02.747724 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde474a9-e888-4f2d-9299-2d2a64ef5d62 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:02.748000 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde474a9-e888-4f2d-9299-2d2a64ef5d62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:02.764346 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde474a9-e888-4f2d-9299-2d2a64ef5d62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:02.766052 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde474a9-e888-4f2d-9299-2d2a64ef5d62 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:02.784079 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1152/2289] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:15:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:15:03.360976 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fccac47-cd5d-406b-b53d-3fd9a9fcdf57 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:03.363396 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-4fccac47-cd5d-406b-b53d-3fd9a9fcdf57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:03.375994 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fccac47-cd5d-406b-b53d-3fd9a9fcdf57 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:03.380935 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b6042e-5b9f-41c3-8f46-1ccda110a593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:03.381703 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b6042e-5b9f-41c3-8f46-1ccda110a593 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:03.381981 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b6042e-5b9f-41c3-8f46-1ccda110a593 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:03.382427 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b6042e-5b9f-41c3-8f46-1ccda110a593 admin admin] PATH_INFO: `/v3/projects/ed818736f2b943c58161a8d5609dfe28` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:03.384085 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b6042e-5b9f-41c3-8f46-1ccda110a593 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed818736f2b943c58161a8d5609dfe28)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:03.385591 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b6042e-5b9f-41c3-8f46-1ccda110a593 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:03.536156 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-65b6042e-5b9f-41c3-8f46-1ccda110a593 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed818736f2b943c58161a8d5609dfe28', 'request_id': 'req-65b6042e-5b9f-41c3-8f46-1ccda110a593'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:03.620818 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-65b6042e-5b9f-41c3-8f46-1ccda110a593 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed818736f2b943c58161a8d5609dfe28', 'request_id': 'req-65b6042e-5b9f-41c3-8f46-1ccda110a593'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:03.635119 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-65b6042e-5b9f-41c3-8f46-1ccda110a593 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed818736f2b943c58161a8d5609dfe28', 'request_id': 'req-65b6042e-5b9f-41c3-8f46-1ccda110a593'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:03.649136 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-65b6042e-5b9f-41c3-8f46-1ccda110a593 admin admin] The token cache is being invalidate because project ed818736f2b943c58161a8d5609dfe28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81650) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:15:03.649431 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-65b6042e-5b9f-41c3-8f46-1ccda110a593 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:03.650098 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-65b6042e-5b9f-41c3-8f46-1ccda110a593 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:03.657411 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1138/2290] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:15:03 2022] DELETE /identity/v3/projects/ed818736f2b943c58161a8d5609dfe28 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:15:03.661421 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfd6b72-ef11-46c0-b9c8-8c6c071a7fab None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:03.662176 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfd6b72-ef11-46c0-b9c8-8c6c071a7fab None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:03.662743 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfd6b72-ef11-46c0-b9c8-8c6c071a7fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:03.688539 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-7dfd6b72-ef11-46c0-b9c8-8c6c071a7fab None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:15:03.689765 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-7dfd6b72-ef11-46c0-b9c8-8c6c071a7fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:03.709054 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:15:03.757902 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1153/2291] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:15:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 17:15:03.765865 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da454b-8616-461b-b69b-75f5cbc26098 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:03.766568 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da454b-8616-461b-b69b-75f5cbc26098 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:03.767173 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da454b-8616-461b-b69b-75f5cbc26098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:03.794675 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-20da454b-8616-461b-b69b-75f5cbc26098 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:15:03.796335 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-20da454b-8616-461b-b69b-75f5cbc26098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:03.809776 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1139/2292] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:15:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:15:03.814264 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfd6b72-ef11-46c0-b9c8-8c6c071a7fab None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:03.825381 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfd6b72-ef11-46c0-b9c8-8c6c071a7fab None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:03.830527 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb6b72-ea9d-4759-8589-36c25dcee648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:03.831252 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb6b72-ea9d-4759-8589-36c25dcee648 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:03.831626 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb6b72-ea9d-4759-8589-36c25dcee648 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:03.832070 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb6b72-ea9d-4759-8589-36c25dcee648 admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:03.832830 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fb6b72-ea9d-4759-8589-36c25dcee648 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:15:03.833280 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fb6b72-ea9d-4759-8589-36c25dcee648 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:03.834989 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fb6b72-ea9d-4759-8589-36c25dcee648 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:03.851021 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1154/2293] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:15:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 17:15:03.852131 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20da454b-8616-461b-b69b-75f5cbc26098 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:03.861944 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20da454b-8616-461b-b69b-75f5cbc26098 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:03.867572 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d7275b-f633-46ec-be35-cf075a0084e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:03.868186 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d7275b-f633-46ec-be35-cf075a0084e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:03.868470 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d7275b-f633-46ec-be35-cf075a0084e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:03.868744 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d7275b-f633-46ec-be35-cf075a0084e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:03.869376 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d7275b-f633-46ec-be35-cf075a0084e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:03.871339 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d7275b-f633-46ec-be35-cf075a0084e7 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:03.969746 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1140/2294] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:15:03 2022] POST /identity/v3/projects => generated 373 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 17:15:03.974085 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb6b72-ea9d-4759-8589-36c25dcee648 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:03.983843 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb6b72-ea9d-4759-8589-36c25dcee648 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:04.051998 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025e9f05-2699-46db-804f-f747124ef19c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:04.058470 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025e9f05-2699-46db-804f-f747124ef19c admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:04.058470 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025e9f05-2699-46db-804f-f747124ef19c admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:04.058470 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025e9f05-2699-46db-804f-f747124ef19c admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:04.058470 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025e9f05-2699-46db-804f-f747124ef19c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:04.058470 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025e9f05-2699-46db-804f-f747124ef19c admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:04.169573 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1155/2295] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:15:03 2022] POST /identity/v3/users => generated 287 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:15:04.173565 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d7275b-f633-46ec-be35-cf075a0084e7 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:04.184462 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d7275b-f633-46ec-be35-cf075a0084e7 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:04.189140 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187fa316-17d2-4f74-8c39-5a8795b1f826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:04.189773 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187fa316-17d2-4f74-8c39-5a8795b1f826 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:04.194601 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187fa316-17d2-4f74-8c39-5a8795b1f826 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:04.194601 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187fa316-17d2-4f74-8c39-5a8795b1f826 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:04.194601 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187fa316-17d2-4f74-8c39-5a8795b1f826 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:04.194601 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187fa316-17d2-4f74-8c39-5a8795b1f826 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:04.202953 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1141/2296] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:15:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:15:04.206833 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025e9f05-2699-46db-804f-f747124ef19c admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:04.223468 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025e9f05-2699-46db-804f-f747124ef19c admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:04.223468 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4ce2dd-562e-437c-b997-f4d435ac20bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:04.223468 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ce2dd-562e-437c-b997-f4d435ac20bb admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:04.223468 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ce2dd-562e-437c-b997-f4d435ac20bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:04.224027 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ce2dd-562e-437c-b997-f4d435ac20bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:04.224123 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4ce2dd-562e-437c-b997-f4d435ac20bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:04.225775 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4ce2dd-562e-437c-b997-f4d435ac20bb admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:04.237228 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1156/2297] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:15:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:15:04.241321 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187fa316-17d2-4f74-8c39-5a8795b1f826 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:04.251518 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187fa316-17d2-4f74-8c39-5a8795b1f826 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:04.258517 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f326415-3138-4f80-bae9-497a02db37b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:04.259237 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f326415-3138-4f80-bae9-497a02db37b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:04.259567 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f326415-3138-4f80-bae9-497a02db37b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:04.259840 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f326415-3138-4f80-bae9-497a02db37b3 admin admin] PATH_INFO: `/v3/projects/f8802bed1f6b4be991ff011f40bf8622/users/ebc5df40911b4186abec99c01e0702a6/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:04.275824 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f326415-3138-4f80-bae9-497a02db37b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8802bed1f6b4be991ff011f40bf8622, user_id=ebc5df40911b4186abec99c01e0702a6, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:04.278584 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f326415-3138-4f80-bae9-497a02db37b3 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:04.377752 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1142/2298] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:15:04 2022] PUT /identity/v3/projects/f8802bed1f6b4be991ff011f40bf8622/users/ebc5df40911b4186abec99c01e0702a6/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:15:04.381366 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a284f067-4950-49e3-ad49-def6a1405805 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:04.381719 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a284f067-4950-49e3-ad49-def6a1405805 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:04.382064 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a284f067-4950-49e3-ad49-def6a1405805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:04.404046 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-a284f067-4950-49e3-ad49-def6a1405805 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:15:04.406017 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-a284f067-4950-49e3-ad49-def6a1405805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:04.465631 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f326415-3138-4f80-bae9-497a02db37b3 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:04.468963 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-5f326415-3138-4f80-bae9-497a02db37b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:04.504923 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1157/2299] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:15:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:15:04.514770 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a284f067-4950-49e3-ad49-def6a1405805 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:04.516234 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-a284f067-4950-49e3-ad49-def6a1405805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:04.540371 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f326415-3138-4f80-bae9-497a02db37b3 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:04.545082 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765fdd00-f426-4a8d-b165-34af8e2227f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:04.545718 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765fdd00-f426-4a8d-b165-34af8e2227f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:04.546006 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765fdd00-f426-4a8d-b165-34af8e2227f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:04.546308 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765fdd00-f426-4a8d-b165-34af8e2227f2 admin admin] PATH_INFO: `/v3/users/d38b8e7cc88841fb8ef93dd4c5c81417` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:04.547484 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765fdd00-f426-4a8d-b165-34af8e2227f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d38b8e7cc88841fb8ef93dd4c5c81417)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:04.548984 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:15:04.549654 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765fdd00-f426-4a8d-b165-34af8e2227f2 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:04.551516 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:15:04.577421 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a284f067-4950-49e3-ad49-def6a1405805 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:04.581242 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8214f0a6-018a-432a-b58c-dc243106987a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:04.581885 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8214f0a6-018a-432a-b58c-dc243106987a service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:04.582164 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8214f0a6-018a-432a-b58c-dc243106987a service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:04.582440 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8214f0a6-018a-432a-b58c-dc243106987a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:04.592701 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8214f0a6-018a-432a-b58c-dc243106987a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:04.594125 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8214f0a6-018a-432a-b58c-dc243106987a service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:04.618746 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1158/2300] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:15:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 17:15:04.813494 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-765fdd00-f426-4a8d-b165-34af8e2227f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd38b8e7cc88841fb8ef93dd4c5c81417'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:04.827669 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-765fdd00-f426-4a8d-b165-34af8e2227f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd38b8e7cc88841fb8ef93dd4c5c81417'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:04.828376 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-765fdd00-f426-4a8d-b165-34af8e2227f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd38b8e7cc88841fb8ef93dd4c5c81417'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:04.934248 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-765fdd00-f426-4a8d-b165-34af8e2227f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd38b8e7cc88841fb8ef93dd4c5c81417'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:04.935375 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-765fdd00-f426-4a8d-b165-34af8e2227f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd38b8e7cc88841fb8ef93dd4c5c81417'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:04.958766 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1143/2301] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:15:04 2022] DELETE /identity/v3/users/d38b8e7cc88841fb8ef93dd4c5c81417 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:15:04.964434 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efaa7cb-da70-4162-82af-f34f38421743 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:04.964879 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efaa7cb-da70-4162-82af-f34f38421743 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:04.965437 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efaa7cb-da70-4162-82af-f34f38421743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:04.987540 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-0efaa7cb-da70-4162-82af-f34f38421743 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:15:04.988814 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-0efaa7cb-da70-4162-82af-f34f38421743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:05.074172 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1159/2302] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:15:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 17:15:05.085588 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765fdd00-f426-4a8d-b165-34af8e2227f2 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:05.087397 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-765fdd00-f426-4a8d-b165-34af8e2227f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:05.159757 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765fdd00-f426-4a8d-b165-34af8e2227f2 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:05.165744 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad5fc5d-7603-47cc-a104-6fcb616c4a55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:05.167056 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad5fc5d-7603-47cc-a104-6fcb616c4a55 service neutron] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:05.167402 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad5fc5d-7603-47cc-a104-6fcb616c4a55 service neutron] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:05.167753 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad5fc5d-7603-47cc-a104-6fcb616c4a55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:05.185147 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad5fc5d-7603-47cc-a104-6fcb616c4a55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:05.186705 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad5fc5d-7603-47cc-a104-6fcb616c4a55 service neutron] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:05.200003 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1144/2303] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:15:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 17:15:05.808695 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa74006-e39a-4611-954f-9053718b6e34 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:05.808999 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa74006-e39a-4611-954f-9053718b6e34 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:05.809257 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa74006-e39a-4611-954f-9053718b6e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:05.828633 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-6fa74006-e39a-4611-954f-9053718b6e34 None None] MFA Rules not processed for user `497f5cef83ff4fe69e00e3ccf703088d`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:15:05.830053 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-6fa74006-e39a-4611-954f-9053718b6e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:05.899562 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad5fc5d-7603-47cc-a104-6fcb616c4a55 service neutron] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:05.901093 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-5ad5fc5d-7603-47cc-a104-6fcb616c4a55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:05.917995 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1160/2304] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:15:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:15:05.922764 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad5fc5d-7603-47cc-a104-6fcb616c4a55 service neutron] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:05.928450 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7553febf-3c0c-4a09-9d05-3cdeb9e28202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:05.929296 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7553febf-3c0c-4a09-9d05-3cdeb9e28202 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:05.929700 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7553febf-3c0c-4a09-9d05-3cdeb9e28202 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:05.930175 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7553febf-3c0c-4a09-9d05-3cdeb9e28202 admin admin] PATH_INFO: `/v3/projects/67b228134d574af0a6c228c145c8737f` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:05.931361 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7553febf-3c0c-4a09-9d05-3cdeb9e28202 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67b228134d574af0a6c228c145c8737f)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:05.933288 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7553febf-3c0c-4a09-9d05-3cdeb9e28202 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:06.092789 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-7553febf-3c0c-4a09-9d05-3cdeb9e28202 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67b228134d574af0a6c228c145c8737f', 'request_id': 'req-7553febf-3c0c-4a09-9d05-3cdeb9e28202'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:06.194707 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-7553febf-3c0c-4a09-9d05-3cdeb9e28202 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67b228134d574af0a6c228c145c8737f', 'request_id': 'req-7553febf-3c0c-4a09-9d05-3cdeb9e28202'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:06.204057 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-7553febf-3c0c-4a09-9d05-3cdeb9e28202 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67b228134d574af0a6c228c145c8737f', 'request_id': 'req-7553febf-3c0c-4a09-9d05-3cdeb9e28202'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:06.223417 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-7553febf-3c0c-4a09-9d05-3cdeb9e28202 admin admin] The token cache is being invalidate because project 67b228134d574af0a6c228c145c8737f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81650) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:15:06.224088 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-7553febf-3c0c-4a09-9d05-3cdeb9e28202 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:06.224671 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-7553febf-3c0c-4a09-9d05-3cdeb9e28202 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:06.226185 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1145/2305] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:15:05 2022] DELETE /identity/v3/projects/67b228134d574af0a6c228c145c8737f => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:15:06.230336 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa74006-e39a-4611-954f-9053718b6e34 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:06.243156 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa74006-e39a-4611-954f-9053718b6e34 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:06.314703 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9651537-7d56-4963-a7ac-49c5cd8672bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:06.315488 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9651537-7d56-4963-a7ac-49c5cd8672bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:06.316056 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9651537-7d56-4963-a7ac-49c5cd8672bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:06.316178 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9651537-7d56-4963-a7ac-49c5cd8672bb admin admin] PATH_INFO: `/v3/users/93ab1d4c30f94403b762ebc2a60d307f` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:06.318414 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9651537-7d56-4963-a7ac-49c5cd8672bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=93ab1d4c30f94403b762ebc2a60d307f)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:06.321117 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9651537-7d56-4963-a7ac-49c5cd8672bb admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:06.366798 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:15:06.572481 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-d9651537-7d56-4963-a7ac-49c5cd8672bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93ab1d4c30f94403b762ebc2a60d307f'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:06.670332 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-d9651537-7d56-4963-a7ac-49c5cd8672bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93ab1d4c30f94403b762ebc2a60d307f'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:06.671053 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-d9651537-7d56-4963-a7ac-49c5cd8672bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93ab1d4c30f94403b762ebc2a60d307f'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:06.671891 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-d9651537-7d56-4963-a7ac-49c5cd8672bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93ab1d4c30f94403b762ebc2a60d307f'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:06.681531 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-d9651537-7d56-4963-a7ac-49c5cd8672bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93ab1d4c30f94403b762ebc2a60d307f'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:06.726594 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1161/2306] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:15:06 2022] DELETE /identity/v3/users/93ab1d4c30f94403b762ebc2a60d307f => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:15:07.561964 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7553febf-3c0c-4a09-9d05-3cdeb9e28202 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:07.563482 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-7553febf-3c0c-4a09-9d05-3cdeb9e28202 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:07.595530 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:15:07.603219 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7553febf-3c0c-4a09-9d05-3cdeb9e28202 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:07.730012 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0490b9d1-801d-4a13-be53-f5e35dc6dde3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:07.730868 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0490b9d1-801d-4a13-be53-f5e35dc6dde3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:07.731333 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0490b9d1-801d-4a13-be53-f5e35dc6dde3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:07.731777 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0490b9d1-801d-4a13-be53-f5e35dc6dde3 admin admin] PATH_INFO: `/v3/projects/aef6084e9e8c4d1dafe4e5b1dda9c276` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:07.733131 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0490b9d1-801d-4a13-be53-f5e35dc6dde3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aef6084e9e8c4d1dafe4e5b1dda9c276)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:07.735317 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0490b9d1-801d-4a13-be53-f5e35dc6dde3 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:07.885065 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-0490b9d1-801d-4a13-be53-f5e35dc6dde3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aef6084e9e8c4d1dafe4e5b1dda9c276', 'request_id': 'req-0490b9d1-801d-4a13-be53-f5e35dc6dde3'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:07.990689 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-0490b9d1-801d-4a13-be53-f5e35dc6dde3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aef6084e9e8c4d1dafe4e5b1dda9c276', 'request_id': 'req-0490b9d1-801d-4a13-be53-f5e35dc6dde3'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:07.997380 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-0490b9d1-801d-4a13-be53-f5e35dc6dde3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aef6084e9e8c4d1dafe4e5b1dda9c276', 'request_id': 'req-0490b9d1-801d-4a13-be53-f5e35dc6dde3'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:08.017423 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-0490b9d1-801d-4a13-be53-f5e35dc6dde3 admin admin] The token cache is being invalidate because project aef6084e9e8c4d1dafe4e5b1dda9c276 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81650) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:15:08.017423 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-0490b9d1-801d-4a13-be53-f5e35dc6dde3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:08.018067 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-0490b9d1-801d-4a13-be53-f5e35dc6dde3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:08.023921 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1146/2307] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:15:07 2022] DELETE /identity/v3/projects/aef6084e9e8c4d1dafe4e5b1dda9c276 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:15:08.024831 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9651537-7d56-4963-a7ac-49c5cd8672bb admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:08.027151 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-d9651537-7d56-4963-a7ac-49c5cd8672bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:08.048268 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9651537-7d56-4963-a7ac-49c5cd8672bb admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:08.168147 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff3c944-266e-4c0a-aa11-cef870b0ab84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:08.174709 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff3c944-266e-4c0a-aa11-cef870b0ab84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:08.174709 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff3c944-266e-4c0a-aa11-cef870b0ab84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:08.174709 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff3c944-266e-4c0a-aa11-cef870b0ab84 admin admin] PATH_INFO: `/v3/users/269417db723f4556bd568ca762115d21` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:08.174709 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff3c944-266e-4c0a-aa11-cef870b0ab84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=269417db723f4556bd568ca762115d21)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:08.174709 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff3c944-266e-4c0a-aa11-cef870b0ab84 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:08.415322 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-0ff3c944-266e-4c0a-aa11-cef870b0ab84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '269417db723f4556bd568ca762115d21'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:08.563642 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-0ff3c944-266e-4c0a-aa11-cef870b0ab84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '269417db723f4556bd568ca762115d21'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:08.563642 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-0ff3c944-266e-4c0a-aa11-cef870b0ab84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '269417db723f4556bd568ca762115d21'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:08.563642 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-0ff3c944-266e-4c0a-aa11-cef870b0ab84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '269417db723f4556bd568ca762115d21'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:08.573942 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-0ff3c944-266e-4c0a-aa11-cef870b0ab84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '269417db723f4556bd568ca762115d21'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:08.605428 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1162/2308] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:15:08 2022] DELETE /identity/v3/users/269417db723f4556bd568ca762115d21 => generated 0 bytes in 581 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:15:09.217441 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0490b9d1-801d-4a13-be53-f5e35dc6dde3 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:09.220025 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-0490b9d1-801d-4a13-be53-f5e35dc6dde3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:09.237715 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0490b9d1-801d-4a13-be53-f5e35dc6dde3 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:09.325437 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2e73e4-7bb0-4ac7-9d9c-39f43d4dd9b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:09.326103 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e73e4-7bb0-4ac7-9d9c-39f43d4dd9b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:09.330781 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e73e4-7bb0-4ac7-9d9c-39f43d4dd9b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:09.330781 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e73e4-7bb0-4ac7-9d9c-39f43d4dd9b9 admin admin] PATH_INFO: `/v3/projects/773023aa750f4b389c0cddace43a0962` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:09.330781 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2e73e4-7bb0-4ac7-9d9c-39f43d4dd9b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=773023aa750f4b389c0cddace43a0962)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:09.330781 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2e73e4-7bb0-4ac7-9d9c-39f43d4dd9b9 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:09.496131 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:15:09.509962 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-1b2e73e4-7bb0-4ac7-9d9c-39f43d4dd9b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '773023aa750f4b389c0cddace43a0962', 'request_id': 'req-1b2e73e4-7bb0-4ac7-9d9c-39f43d4dd9b9'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:09.636231 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-1b2e73e4-7bb0-4ac7-9d9c-39f43d4dd9b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '773023aa750f4b389c0cddace43a0962', 'request_id': 'req-1b2e73e4-7bb0-4ac7-9d9c-39f43d4dd9b9'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:09.643964 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-1b2e73e4-7bb0-4ac7-9d9c-39f43d4dd9b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '773023aa750f4b389c0cddace43a0962', 'request_id': 'req-1b2e73e4-7bb0-4ac7-9d9c-39f43d4dd9b9'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:09.662810 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-1b2e73e4-7bb0-4ac7-9d9c-39f43d4dd9b9 admin admin] The token cache is being invalidate because project 773023aa750f4b389c0cddace43a0962 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81650) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:15:09.663242 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-1b2e73e4-7bb0-4ac7-9d9c-39f43d4dd9b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:09.663922 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-1b2e73e4-7bb0-4ac7-9d9c-39f43d4dd9b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:09.665714 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1147/2309] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:15:09 2022] DELETE /identity/v3/projects/773023aa750f4b389c0cddace43a0962 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:15:09.671504 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d9952-782a-482c-be8e-8f50b874ccfe None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:09.672043 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d9952-782a-482c-be8e-8f50b874ccfe None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:09.672043 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d9952-782a-482c-be8e-8f50b874ccfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:09.701522 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-324d9952-782a-482c-be8e-8f50b874ccfe None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:15:09.702796 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-324d9952-782a-482c-be8e-8f50b874ccfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:09.725106 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:15:09.880851 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19cecd0-8721-48d8-950d-0ed6fa93400f None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:09.881218 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19cecd0-8721-48d8-950d-0ed6fa93400f None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:09.881932 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19cecd0-8721-48d8-950d-0ed6fa93400f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:09.882056 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1163/2310] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:15:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:15:09.933121 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-a19cecd0-8721-48d8-950d-0ed6fa93400f None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:15:09.934362 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-a19cecd0-8721-48d8-950d-0ed6fa93400f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:09.956886 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1148/2311] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:15:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 17:15:09.958921 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324d9952-782a-482c-be8e-8f50b874ccfe None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:09.994159 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324d9952-782a-482c-be8e-8f50b874ccfe None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:10.019333 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c62ac74-8269-4b92-b7cc-57801a79db76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:10.020560 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c62ac74-8269-4b92-b7cc-57801a79db76 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:10.021002 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c62ac74-8269-4b92-b7cc-57801a79db76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:10.023010 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c62ac74-8269-4b92-b7cc-57801a79db76 admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:10.023010 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c62ac74-8269-4b92-b7cc-57801a79db76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:15:10.023010 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c62ac74-8269-4b92-b7cc-57801a79db76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:10.029912 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c62ac74-8269-4b92-b7cc-57801a79db76 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:10.059855 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1164/2312] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:15:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 17:15:10.060731 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19cecd0-8721-48d8-950d-0ed6fa93400f None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:10.071710 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19cecd0-8721-48d8-950d-0ed6fa93400f None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:10.076939 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56170623-c60a-4137-be78-333c093d5f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:10.081146 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56170623-c60a-4137-be78-333c093d5f2b admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:10.081466 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56170623-c60a-4137-be78-333c093d5f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:10.081761 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56170623-c60a-4137-be78-333c093d5f2b admin admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:10.082317 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56170623-c60a-4137-be78-333c093d5f2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:10.085397 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56170623-c60a-4137-be78-333c093d5f2b admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:10.194252 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1149/2313] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:15:10 2022] POST /identity/v3/projects => generated 373 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:15:10.199090 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c62ac74-8269-4b92-b7cc-57801a79db76 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:10.212295 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c62ac74-8269-4b92-b7cc-57801a79db76 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:10.297842 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0680a3-2563-457e-8396-97d49355e074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:10.298537 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0680a3-2563-457e-8396-97d49355e074 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:10.298884 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0680a3-2563-457e-8396-97d49355e074 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:10.299249 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0680a3-2563-457e-8396-97d49355e074 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:10.299885 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0680a3-2563-457e-8396-97d49355e074 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:10.302488 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0680a3-2563-457e-8396-97d49355e074 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:10.464231 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1165/2314] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:15:10 2022] POST /identity/v3/users => generated 287 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:15:10.468681 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56170623-c60a-4137-be78-333c093d5f2b admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:10.483722 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56170623-c60a-4137-be78-333c093d5f2b admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:10.492565 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc916c3-2716-4ee1-9ad8-d8e527d332a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:10.494109 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc916c3-2716-4ee1-9ad8-d8e527d332a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:10.494404 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc916c3-2716-4ee1-9ad8-d8e527d332a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:10.494819 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc916c3-2716-4ee1-9ad8-d8e527d332a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:10.500232 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc916c3-2716-4ee1-9ad8-d8e527d332a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:10.502204 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc916c3-2716-4ee1-9ad8-d8e527d332a7 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:10.527498 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1150/2315] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:15:10 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:15:10.527846 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0680a3-2563-457e-8396-97d49355e074 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:10.556005 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0680a3-2563-457e-8396-97d49355e074 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:10.575246 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f98f6e-aabf-475a-bfd0-a2f93300e88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:10.576187 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f98f6e-aabf-475a-bfd0-a2f93300e88e admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:10.578874 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f98f6e-aabf-475a-bfd0-a2f93300e88e admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:10.580206 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f98f6e-aabf-475a-bfd0-a2f93300e88e admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:10.581029 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f98f6e-aabf-475a-bfd0-a2f93300e88e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:10.584760 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f98f6e-aabf-475a-bfd0-a2f93300e88e admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:10.610056 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1166/2316] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:15:10 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:15:10.611102 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc916c3-2716-4ee1-9ad8-d8e527d332a7 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:10.625563 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc916c3-2716-4ee1-9ad8-d8e527d332a7 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:10.639466 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5726d4dd-1172-4adb-9e5f-ca29e8972abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:10.640356 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5726d4dd-1172-4adb-9e5f-ca29e8972abd admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:10.640801 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5726d4dd-1172-4adb-9e5f-ca29e8972abd admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:10.641249 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5726d4dd-1172-4adb-9e5f-ca29e8972abd admin admin] PATH_INFO: `/v3/projects/8066561e145b47b5aaec87b7d0643272/users/be74b483a7ea45a9b002dbab58ea0dba/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:10.694885 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5726d4dd-1172-4adb-9e5f-ca29e8972abd admin admin] RBAC: Authorizing `identity:create_grant(project_id=8066561e145b47b5aaec87b7d0643272, user_id=be74b483a7ea45a9b002dbab58ea0dba, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:10.698527 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5726d4dd-1172-4adb-9e5f-ca29e8972abd admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:10.800350 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1151/2317] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:15:10 2022] PUT /identity/v3/projects/8066561e145b47b5aaec87b7d0643272/users/be74b483a7ea45a9b002dbab58ea0dba/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:15:10.806267 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f611a3-3c28-455f-8ab0-5772ed3ce3a4 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:10.807352 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f611a3-3c28-455f-8ab0-5772ed3ce3a4 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:10.807934 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f611a3-3c28-455f-8ab0-5772ed3ce3a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:10.840332 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-48f611a3-3c28-455f-8ab0-5772ed3ce3a4 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:15:10.841880 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-48f611a3-3c28-455f-8ab0-5772ed3ce3a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:11.033664 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1167/2318] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:15:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 17:15:11.076264 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5726d4dd-1172-4adb-9e5f-ca29e8972abd admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:11.092680 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-5726d4dd-1172-4adb-9e5f-ca29e8972abd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:11.150197 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f611a3-3c28-455f-8ab0-5772ed3ce3a4 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:11.151913 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-48f611a3-3c28-455f-8ab0-5772ed3ce3a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:11.173491 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f611a3-3c28-455f-8ab0-5772ed3ce3a4 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:11.189868 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8c5ca6-5b5c-421b-a261-99c76931b05d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:11.192368 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c5ca6-5b5c-421b-a261-99c76931b05d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:11.194342 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5726d4dd-1172-4adb-9e5f-ca29e8972abd admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:11.194960 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c5ca6-5b5c-421b-a261-99c76931b05d admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:11.195358 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c5ca6-5b5c-421b-a261-99c76931b05d admin admin] PATH_INFO: `/v3/users/4eab2fcc430b4300b31e480b65d82ac3` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:11.196997 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8c5ca6-5b5c-421b-a261-99c76931b05d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4eab2fcc430b4300b31e480b65d82ac3)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:11.199097 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d56d72-25a1-481b-a741-885c9cf37014 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:11.199736 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d56d72-25a1-481b-a741-885c9cf37014 service neutron] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:11.199866 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8c5ca6-5b5c-421b-a261-99c76931b05d admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:11.200021 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d56d72-25a1-481b-a741-885c9cf37014 service neutron] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:11.200302 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d56d72-25a1-481b-a741-885c9cf37014 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:11.213393 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d56d72-25a1-481b-a741-885c9cf37014 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:11.215245 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d56d72-25a1-481b-a741-885c9cf37014 service neutron] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:11.241520 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1152/2319] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:15:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:15:11.274053 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:15:11.471022 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-bf8c5ca6-5b5c-421b-a261-99c76931b05d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4eab2fcc430b4300b31e480b65d82ac3'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:11.582237 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-bf8c5ca6-5b5c-421b-a261-99c76931b05d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4eab2fcc430b4300b31e480b65d82ac3'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:11.584471 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-bf8c5ca6-5b5c-421b-a261-99c76931b05d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4eab2fcc430b4300b31e480b65d82ac3'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:11.585738 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-bf8c5ca6-5b5c-421b-a261-99c76931b05d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4eab2fcc430b4300b31e480b65d82ac3'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:11.605115 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-bf8c5ca6-5b5c-421b-a261-99c76931b05d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4eab2fcc430b4300b31e480b65d82ac3'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:11.649460 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1168/2320] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:15:11 2022] DELETE /identity/v3/users/4eab2fcc430b4300b31e480b65d82ac3 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:15:11.651214 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb65850-4648-47dc-9969-3a26a084adc8 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:11.651651 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb65850-4648-47dc-9969-3a26a084adc8 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:11.651777 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb65850-4648-47dc-9969-3a26a084adc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:11.693315 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-ffb65850-4648-47dc-9969-3a26a084adc8 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:15:11.694840 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-ffb65850-4648-47dc-9969-3a26a084adc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:11.793279 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1153/2321] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:15:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:15:11.822376 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8c5ca6-5b5c-421b-a261-99c76931b05d admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:11.823881 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-bf8c5ca6-5b5c-421b-a261-99c76931b05d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:11.993562 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e5582d-66dd-4d7c-8aa5-9cd2d5e5886a None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:11.994153 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e5582d-66dd-4d7c-8aa5-9cd2d5e5886a None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:11.994756 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e5582d-66dd-4d7c-8aa5-9cd2d5e5886a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:11.999153 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8c5ca6-5b5c-421b-a261-99c76931b05d admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:12.005664 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:15:12.008314 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0299da5e-9f7a-4358-b874-96d0b7734d55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:12.009727 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0299da5e-9f7a-4358-b874-96d0b7734d55 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:12.011415 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0299da5e-9f7a-4358-b874-96d0b7734d55 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:12.013725 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0299da5e-9f7a-4358-b874-96d0b7734d55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:12.049741 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0299da5e-9f7a-4358-b874-96d0b7734d55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:12.052612 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0299da5e-9f7a-4358-b874-96d0b7734d55 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:12.061067 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-72e5582d-66dd-4d7c-8aa5-9cd2d5e5886a None None] MFA Rules not processed for user `ebc5df40911b4186abec99c01e0702a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:15:12.062726 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-72e5582d-66dd-4d7c-8aa5-9cd2d5e5886a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:12.080822 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1169/2322] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:15:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:15:12.184488 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1154/2323] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:15:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 17:15:12.188649 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0299da5e-9f7a-4358-b874-96d0b7734d55 service neutron] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:12.190411 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-0299da5e-9f7a-4358-b874-96d0b7734d55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:12.219948 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0299da5e-9f7a-4358-b874-96d0b7734d55 service neutron] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:12.226645 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4547b55a-0bbf-4f7c-8be7-e1e3c3211715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:12.227604 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4547b55a-0bbf-4f7c-8be7-e1e3c3211715 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:12.228181 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4547b55a-0bbf-4f7c-8be7-e1e3c3211715 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:12.228658 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4547b55a-0bbf-4f7c-8be7-e1e3c3211715 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:12.229597 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4547b55a-0bbf-4f7c-8be7-e1e3c3211715 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:12.232551 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4547b55a-0bbf-4f7c-8be7-e1e3c3211715 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:12.367408 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1170/2324] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:15:12 2022] POST /identity/v3/projects => generated 371 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:15:12.371241 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e5582d-66dd-4d7c-8aa5-9cd2d5e5886a None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:12.382709 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e5582d-66dd-4d7c-8aa5-9cd2d5e5886a None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:12.462017 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4761d5ee-ac84-44ad-bb23-8a2d54eb0d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:12.462717 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4761d5ee-ac84-44ad-bb23-8a2d54eb0d8f admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:12.463099 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4761d5ee-ac84-44ad-bb23-8a2d54eb0d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:12.463646 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4761d5ee-ac84-44ad-bb23-8a2d54eb0d8f admin admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:12.464394 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4761d5ee-ac84-44ad-bb23-8a2d54eb0d8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:12.466551 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4761d5ee-ac84-44ad-bb23-8a2d54eb0d8f admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:12.563434 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1155/2325] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:15:12 2022] POST /identity/v3/users => generated 292 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:15:12.570195 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4547b55a-0bbf-4f7c-8be7-e1e3c3211715 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:12.587671 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4547b55a-0bbf-4f7c-8be7-e1e3c3211715 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:12.600455 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47d9792-b9d3-48d3-86c4-a38e6794d211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:12.601366 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47d9792-b9d3-48d3-86c4-a38e6794d211 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:12.601833 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47d9792-b9d3-48d3-86c4-a38e6794d211 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:12.602328 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47d9792-b9d3-48d3-86c4-a38e6794d211 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:12.603308 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47d9792-b9d3-48d3-86c4-a38e6794d211 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:12.605846 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47d9792-b9d3-48d3-86c4-a38e6794d211 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:12.622941 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1171/2326] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:15:12 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:15:12.627503 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4761d5ee-ac84-44ad-bb23-8a2d54eb0d8f admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:12.652344 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4761d5ee-ac84-44ad-bb23-8a2d54eb0d8f admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:12.656641 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a542ade-7b4b-41b4-9b95-c02bf4c9220d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:12.657620 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a542ade-7b4b-41b4-9b95-c02bf4c9220d admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:12.658071 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a542ade-7b4b-41b4-9b95-c02bf4c9220d admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:12.658681 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a542ade-7b4b-41b4-9b95-c02bf4c9220d admin admin] PATH_INFO: `/v3/projects/5db98e82fc6f4234a6b7bc57139ec073/users/6af30f5ad30b433983e90592fd3f4d8c/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:12.679813 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a542ade-7b4b-41b4-9b95-c02bf4c9220d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5db98e82fc6f4234a6b7bc57139ec073, user_id=6af30f5ad30b433983e90592fd3f4d8c, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:12.683309 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a542ade-7b4b-41b4-9b95-c02bf4c9220d admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:12.774161 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1156/2327] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:15:12 2022] PUT /identity/v3/projects/5db98e82fc6f4234a6b7bc57139ec073/users/6af30f5ad30b433983e90592fd3f4d8c/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:15:12.780376 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47d9792-b9d3-48d3-86c4-a38e6794d211 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:12.796103 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47d9792-b9d3-48d3-86c4-a38e6794d211 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:12.855866 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a542ade-7b4b-41b4-9b95-c02bf4c9220d admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:12.859908 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-7a542ade-7b4b-41b4-9b95-c02bf4c9220d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:12.873998 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a292ee-c7f7-48bc-ba57-64f9f42f81ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:12.874855 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a292ee-c7f7-48bc-ba57-64f9f42f81ba admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:12.878595 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a292ee-c7f7-48bc-ba57-64f9f42f81ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:12.878595 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a292ee-c7f7-48bc-ba57-64f9f42f81ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:12.878595 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a292ee-c7f7-48bc-ba57-64f9f42f81ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:12.878595 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a292ee-c7f7-48bc-ba57-64f9f42f81ba admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:12.885460 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:15:12.890934 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1172/2328] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:15:12 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:15:12.900477 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a292ee-c7f7-48bc-ba57-64f9f42f81ba admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:12.915916 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a292ee-c7f7-48bc-ba57-64f9f42f81ba admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:12.922827 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c8ec96-b02d-4c84-9481-c33d7e42c409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:12.924128 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c8ec96-b02d-4c84-9481-c33d7e42c409 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:12.924516 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c8ec96-b02d-4c84-9481-c33d7e42c409 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:12.924862 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c8ec96-b02d-4c84-9481-c33d7e42c409 admin admin] PATH_INFO: `/v3/projects/5db98e82fc6f4234a6b7bc57139ec073/users/6af30f5ad30b433983e90592fd3f4d8c/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:12.927852 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c8ec96-b02d-4c84-9481-c33d7e42c409 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5db98e82fc6f4234a6b7bc57139ec073, user_id=6af30f5ad30b433983e90592fd3f4d8c, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:12.935811 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c8ec96-b02d-4c84-9481-c33d7e42c409 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:12.950114 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1173/2329] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:15:12 2022] PUT /identity/v3/projects/5db98e82fc6f4234a6b7bc57139ec073/users/6af30f5ad30b433983e90592fd3f4d8c/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:15:12.968831 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a542ade-7b4b-41b4-9b95-c02bf4c9220d admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:13.059503 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886c9cd1-bc81-4cca-94d7-91e5d782c671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:13.063660 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886c9cd1-bc81-4cca-94d7-91e5d782c671 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:13.065653 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886c9cd1-bc81-4cca-94d7-91e5d782c671 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:13.066024 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886c9cd1-bc81-4cca-94d7-91e5d782c671 admin admin] PATH_INFO: `/v3/projects/9b85847639bf4d80b2fff671fba22e6c` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:13.067605 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886c9cd1-bc81-4cca-94d7-91e5d782c671 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b85847639bf4d80b2fff671fba22e6c)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:13.069735 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886c9cd1-bc81-4cca-94d7-91e5d782c671 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:13.108874 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:15:13.221117 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-886c9cd1-bc81-4cca-94d7-91e5d782c671 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b85847639bf4d80b2fff671fba22e6c', 'request_id': 'req-886c9cd1-bc81-4cca-94d7-91e5d782c671'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:13.307867 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-886c9cd1-bc81-4cca-94d7-91e5d782c671 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b85847639bf4d80b2fff671fba22e6c', 'request_id': 'req-886c9cd1-bc81-4cca-94d7-91e5d782c671'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:13.320557 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-886c9cd1-bc81-4cca-94d7-91e5d782c671 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b85847639bf4d80b2fff671fba22e6c', 'request_id': 'req-886c9cd1-bc81-4cca-94d7-91e5d782c671'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:13.353035 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-886c9cd1-bc81-4cca-94d7-91e5d782c671 admin admin] The token cache is being invalidate because project 9b85847639bf4d80b2fff671fba22e6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81650) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:15:13.353366 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-886c9cd1-bc81-4cca-94d7-91e5d782c671 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:13.354068 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-886c9cd1-bc81-4cca-94d7-91e5d782c671 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:13.355705 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1157/2330] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:15:12 2022] DELETE /identity/v3/projects/9b85847639bf4d80b2fff671fba22e6c => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:15:13.359780 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c8ec96-b02d-4c84-9481-c33d7e42c409 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:13.362025 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-86c8ec96-b02d-4c84-9481-c33d7e42c409 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:13.382740 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c8ec96-b02d-4c84-9481-c33d7e42c409 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:13.471728 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7962a1af-ba7c-464f-aca7-dec940e4a69b None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:13.472004 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7962a1af-ba7c-464f-aca7-dec940e4a69b None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:13.472293 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7962a1af-ba7c-464f-aca7-dec940e4a69b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:13.507808 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27fac06-61a2-4b07-b618-5bfb896b6e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:13.508412 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27fac06-61a2-4b07-b618-5bfb896b6e30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:13.508706 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27fac06-61a2-4b07-b618-5bfb896b6e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:13.508983 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27fac06-61a2-4b07-b618-5bfb896b6e30 admin admin] PATH_INFO: `/v3/users/6021ae72beae4c939a13fbd2b62eaf5c` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:13.511154 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27fac06-61a2-4b07-b618-5bfb896b6e30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6021ae72beae4c939a13fbd2b62eaf5c)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:13.514864 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27fac06-61a2-4b07-b618-5bfb896b6e30 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:13.538011 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-7962a1af-ba7c-464f-aca7-dec940e4a69b None None] MFA Rules not processed for user `be74b483a7ea45a9b002dbab58ea0dba`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:15:13.539754 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-7962a1af-ba7c-464f-aca7-dec940e4a69b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:13.662393 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1158/2331] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:15:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:15:13.664420 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7962a1af-ba7c-464f-aca7-dec940e4a69b None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:13.666021 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-7962a1af-ba7c-464f-aca7-dec940e4a69b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:13.687134 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7962a1af-ba7c-464f-aca7-dec940e4a69b None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:13.693068 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e646b4e-3ba5-4587-aaa4-d892a626d71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:13.693696 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e646b4e-3ba5-4587-aaa4-d892a626d71e admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:13.694156 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e646b4e-3ba5-4587-aaa4-d892a626d71e admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:13.694474 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e646b4e-3ba5-4587-aaa4-d892a626d71e admin admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:13.695590 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e646b4e-3ba5-4587-aaa4-d892a626d71e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:13.697520 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e646b4e-3ba5-4587-aaa4-d892a626d71e admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:13.813581 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e27fac06-61a2-4b07-b618-5bfb896b6e30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6021ae72beae4c939a13fbd2b62eaf5c'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:13.827412 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1159/2332] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:15:13 2022] POST /identity/v3/projects => generated 373 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:15:13.828592 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e646b4e-3ba5-4587-aaa4-d892a626d71e admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:13.902638 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e27fac06-61a2-4b07-b618-5bfb896b6e30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6021ae72beae4c939a13fbd2b62eaf5c'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:13.903952 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e27fac06-61a2-4b07-b618-5bfb896b6e30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6021ae72beae4c939a13fbd2b62eaf5c'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:13.904871 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e27fac06-61a2-4b07-b618-5bfb896b6e30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6021ae72beae4c939a13fbd2b62eaf5c'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:13.919991 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e27fac06-61a2-4b07-b618-5bfb896b6e30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6021ae72beae4c939a13fbd2b62eaf5c'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:13.965822 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e646b4e-3ba5-4587-aaa4-d892a626d71e admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:13.973865 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1174/2333] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:15:13 2022] DELETE /identity/v3/users/6021ae72beae4c939a13fbd2b62eaf5c => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:15:14.048807 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78fd471-4c49-4cfa-a2ee-af18c5e4cace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:14.051623 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78fd471-4c49-4cfa-a2ee-af18c5e4cace admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:14.051623 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78fd471-4c49-4cfa-a2ee-af18c5e4cace admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:14.051623 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78fd471-4c49-4cfa-a2ee-af18c5e4cace admin admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:14.056763 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78fd471-4c49-4cfa-a2ee-af18c5e4cace admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:14.056763 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78fd471-4c49-4cfa-a2ee-af18c5e4cace admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:14.149179 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1160/2334] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:15:13 2022] POST /identity/v3/users => generated 293 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:15:14.153803 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27fac06-61a2-4b07-b618-5bfb896b6e30 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:14.155392 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-e27fac06-61a2-4b07-b618-5bfb896b6e30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:14.163976 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:15:14.185386 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27fac06-61a2-4b07-b618-5bfb896b6e30 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:14.206217 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c75a39-2fa3-4ada-a420-03224911c85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:14.208849 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c75a39-2fa3-4ada-a420-03224911c85b admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:14.213702 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c75a39-2fa3-4ada-a420-03224911c85b admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:14.214393 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c75a39-2fa3-4ada-a420-03224911c85b admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:14.216718 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c75a39-2fa3-4ada-a420-03224911c85b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:14.219183 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c75a39-2fa3-4ada-a420-03224911c85b admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:14.242329 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1175/2335] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:15:14 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:15:14.248310 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78fd471-4c49-4cfa-a2ee-af18c5e4cace admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:14.269159 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78fd471-4c49-4cfa-a2ee-af18c5e4cace admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:14.280574 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5168232e-af5b-4498-9f83-8164ffd9c0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:14.281264 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5168232e-af5b-4498-9f83-8164ffd9c0cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:14.281765 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5168232e-af5b-4498-9f83-8164ffd9c0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:14.282048 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5168232e-af5b-4498-9f83-8164ffd9c0cc admin admin] PATH_INFO: `/v3/projects/5260f07fd435401da7acf7778026585d/users/284490df3764411c9aa15edbf78b1541/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:14.308693 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5168232e-af5b-4498-9f83-8164ffd9c0cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=5260f07fd435401da7acf7778026585d, user_id=284490df3764411c9aa15edbf78b1541, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:14.312910 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5168232e-af5b-4498-9f83-8164ffd9c0cc admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:14.396259 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1161/2336] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:15:14 2022] PUT /identity/v3/projects/5260f07fd435401da7acf7778026585d/users/284490df3764411c9aa15edbf78b1541/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:15:14.397442 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c75a39-2fa3-4ada-a420-03224911c85b admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:14.413062 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c75a39-2fa3-4ada-a420-03224911c85b admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:14.512693 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:15:14.556448 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8489f527-15ee-4618-ba88-2a82db69d767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:14.557921 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8489f527-15ee-4618-ba88-2a82db69d767 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:14.559631 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8489f527-15ee-4618-ba88-2a82db69d767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:14.560090 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8489f527-15ee-4618-ba88-2a82db69d767 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:14.566024 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8489f527-15ee-4618-ba88-2a82db69d767 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:14.578925 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8489f527-15ee-4618-ba88-2a82db69d767 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:14.610149 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1176/2337] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:15:14 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 17:15:14.614456 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5168232e-af5b-4498-9f83-8164ffd9c0cc admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:14.629990 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5168232e-af5b-4498-9f83-8164ffd9c0cc admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:14.635886 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d2705b-6fb0-4400-ba4d-0c049d0c1f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:14.637137 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2705b-6fb0-4400-ba4d-0c049d0c1f5f admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:14.637583 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2705b-6fb0-4400-ba4d-0c049d0c1f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:14.638065 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2705b-6fb0-4400-ba4d-0c049d0c1f5f admin admin] PATH_INFO: `/v3/projects/5260f07fd435401da7acf7778026585d/users/284490df3764411c9aa15edbf78b1541/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:14.640321 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d2705b-6fb0-4400-ba4d-0c049d0c1f5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5260f07fd435401da7acf7778026585d, user_id=284490df3764411c9aa15edbf78b1541, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:14.643874 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d2705b-6fb0-4400-ba4d-0c049d0c1f5f admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:14.664791 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1162/2338] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:15:14 2022] PUT /identity/v3/projects/5260f07fd435401da7acf7778026585d/users/284490df3764411c9aa15edbf78b1541/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:15:14.980371 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8489f527-15ee-4618-ba88-2a82db69d767 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:14.980851 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-8489f527-15ee-4618-ba88-2a82db69d767 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:15.001721 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8489f527-15ee-4618-ba88-2a82db69d767 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:15.128774 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f975849-a9f2-40af-8b18-d178019e4c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:15.130120 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f975849-a9f2-40af-8b18-d178019e4c7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:15.130740 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f975849-a9f2-40af-8b18-d178019e4c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:15.131187 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f975849-a9f2-40af-8b18-d178019e4c7b admin admin] PATH_INFO: `/v3/projects/3b97f3d88ec94b88907e89e4f7e1edaa` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:15.140674 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f975849-a9f2-40af-8b18-d178019e4c7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b97f3d88ec94b88907e89e4f7e1edaa)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:15.140674 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f975849-a9f2-40af-8b18-d178019e4c7b admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:15.289143 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-2f975849-a9f2-40af-8b18-d178019e4c7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b97f3d88ec94b88907e89e4f7e1edaa', 'request_id': 'req-2f975849-a9f2-40af-8b18-d178019e4c7b'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:15.305712 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-2f975849-a9f2-40af-8b18-d178019e4c7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b97f3d88ec94b88907e89e4f7e1edaa', 'request_id': 'req-2f975849-a9f2-40af-8b18-d178019e4c7b'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:15.413576 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-2f975849-a9f2-40af-8b18-d178019e4c7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b97f3d88ec94b88907e89e4f7e1edaa', 'request_id': 'req-2f975849-a9f2-40af-8b18-d178019e4c7b'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:15.429835 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-2f975849-a9f2-40af-8b18-d178019e4c7b admin admin] The token cache is being invalidate because project 3b97f3d88ec94b88907e89e4f7e1edaa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:15:15.430136 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-2f975849-a9f2-40af-8b18-d178019e4c7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:15.432465 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-2f975849-a9f2-40af-8b18-d178019e4c7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:15.475279 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1177/2339] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:15:14 2022] DELETE /identity/v3/projects/3b97f3d88ec94b88907e89e4f7e1edaa => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:15:15.481384 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b015928-eebc-4dbc-8e52-c9553969e064 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:15.481673 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b015928-eebc-4dbc-8e52-c9553969e064 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:15.481954 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b015928-eebc-4dbc-8e52-c9553969e064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:15.513108 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-7b015928-eebc-4dbc-8e52-c9553969e064 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:15:15.514433 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-7b015928-eebc-4dbc-8e52-c9553969e064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:15.601588 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1163/2340] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:15:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:15:15.609422 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deb4d6e-c346-430b-9aca-84926796c4f1 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:15.609959 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deb4d6e-c346-430b-9aca-84926796c4f1 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:15.610400 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deb4d6e-c346-430b-9aca-84926796c4f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:15.635781 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-3deb4d6e-c346-430b-9aca-84926796c4f1 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:15:15.637364 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-3deb4d6e-c346-430b-9aca-84926796c4f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:15.657739 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1178/2341] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:15:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:15:15.663627 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b015928-eebc-4dbc-8e52-c9553969e064 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:15.684484 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b015928-eebc-4dbc-8e52-c9553969e064 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:15.700947 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33086f77-fdd0-4375-85e1-b2bcd787bcb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:15.701969 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33086f77-fdd0-4375-85e1-b2bcd787bcb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:15.703630 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33086f77-fdd0-4375-85e1-b2bcd787bcb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:15.704246 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33086f77-fdd0-4375-85e1-b2bcd787bcb7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:15.705131 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33086f77-fdd0-4375-85e1-b2bcd787bcb7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:15:15.705602 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33086f77-fdd0-4375-85e1-b2bcd787bcb7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:15.719234 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33086f77-fdd0-4375-85e1-b2bcd787bcb7 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:15.745981 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1164/2342] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:15:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 17:15:15.747237 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deb4d6e-c346-430b-9aca-84926796c4f1 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:15.761445 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deb4d6e-c346-430b-9aca-84926796c4f1 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:15.777081 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88503d0c-c41a-4017-aa83-4e51d5df91e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:15.777721 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88503d0c-c41a-4017-aa83-4e51d5df91e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:15.778000 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88503d0c-c41a-4017-aa83-4e51d5df91e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:15.778273 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88503d0c-c41a-4017-aa83-4e51d5df91e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:15.778931 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88503d0c-c41a-4017-aa83-4e51d5df91e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:15.791797 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88503d0c-c41a-4017-aa83-4e51d5df91e5 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:15.904748 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1179/2343] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:15:15 2022] POST /identity/v3/projects => generated 365 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:15:15.909203 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33086f77-fdd0-4375-85e1-b2bcd787bcb7 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:15.924614 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33086f77-fdd0-4375-85e1-b2bcd787bcb7 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:16.073706 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c09763f-601f-4662-8fdf-dbbffc666de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:16.093520 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c09763f-601f-4662-8fdf-dbbffc666de3 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:16.094158 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c09763f-601f-4662-8fdf-dbbffc666de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:16.094649 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c09763f-601f-4662-8fdf-dbbffc666de3 admin admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:16.095500 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c09763f-601f-4662-8fdf-dbbffc666de3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:16.099408 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c09763f-601f-4662-8fdf-dbbffc666de3 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:16.112432 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:15:16.253154 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88503d0c-c41a-4017-aa83-4e51d5df91e5 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:16.277721 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1165/2344] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:15:15 2022] POST /identity/v3/users => generated 283 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:15:16.284792 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88503d0c-c41a-4017-aa83-4e51d5df91e5 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:16.291357 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d4d7e9-fea1-4e2f-ad85-2817615e7dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:16.293187 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d4d7e9-fea1-4e2f-ad85-2817615e7dae admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:16.293666 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d4d7e9-fea1-4e2f-ad85-2817615e7dae admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:16.294190 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d4d7e9-fea1-4e2f-ad85-2817615e7dae admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:16.295271 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d4d7e9-fea1-4e2f-ad85-2817615e7dae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:16.297412 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d4d7e9-fea1-4e2f-ad85-2817615e7dae admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:16.323765 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c09763f-601f-4662-8fdf-dbbffc666de3 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:16.324088 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1180/2345] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:15:16 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:15:16.337013 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c09763f-601f-4662-8fdf-dbbffc666de3 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:16.343083 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfba1ff-217f-45ed-964c-094d0eb4065d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:16.344443 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfba1ff-217f-45ed-964c-094d0eb4065d admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:16.344863 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfba1ff-217f-45ed-964c-094d0eb4065d admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:16.345305 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfba1ff-217f-45ed-964c-094d0eb4065d admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:16.346046 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfba1ff-217f-45ed-964c-094d0eb4065d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:16.348001 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfba1ff-217f-45ed-964c-094d0eb4065d admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:16.361858 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1166/2346] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:15:16 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:15:16.367790 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d4d7e9-fea1-4e2f-ad85-2817615e7dae admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:16.385271 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d4d7e9-fea1-4e2f-ad85-2817615e7dae admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:16.395227 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf0bdce-fbf7-44c4-9761-8a8dec5b5073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:16.407593 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf0bdce-fbf7-44c4-9761-8a8dec5b5073 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:16.407593 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf0bdce-fbf7-44c4-9761-8a8dec5b5073 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:16.407593 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf0bdce-fbf7-44c4-9761-8a8dec5b5073 admin admin] PATH_INFO: `/v3/projects/68cc38c90f6641adb3c815f20f18f6f7/users/9cb29abd135642fea4d19496d4a12499/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:16.463295 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf0bdce-fbf7-44c4-9761-8a8dec5b5073 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68cc38c90f6641adb3c815f20f18f6f7, user_id=9cb29abd135642fea4d19496d4a12499, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:16.467611 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf0bdce-fbf7-44c4-9761-8a8dec5b5073 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:16.565267 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1181/2347] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:15:16 2022] PUT /identity/v3/projects/68cc38c90f6641adb3c815f20f18f6f7/users/9cb29abd135642fea4d19496d4a12499/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:15:16.578093 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2207c3dd-7186-4afb-864a-de1e77dd00cf None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:16.578721 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2207c3dd-7186-4afb-864a-de1e77dd00cf None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:16.579184 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2207c3dd-7186-4afb-864a-de1e77dd00cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:16.623750 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-2207c3dd-7186-4afb-864a-de1e77dd00cf None None] MFA Rules not processed for user `9cb29abd135642fea4d19496d4a12499`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:15:16.625381 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-2207c3dd-7186-4afb-864a-de1e77dd00cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:16.777441 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1167/2348] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:15:16 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:15:16.789189 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf0bdce-fbf7-44c4-9761-8a8dec5b5073 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:16.827359 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:15:16.886366 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf0bdce-fbf7-44c4-9761-8a8dec5b5073 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:16.891221 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d11506e-458e-49ea-9d25-fdbb638ca46b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:16.891851 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d11506e-458e-49ea-9d25-fdbb638ca46b admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:16.892163 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d11506e-458e-49ea-9d25-fdbb638ca46b admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:16.892401 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d11506e-458e-49ea-9d25-fdbb638ca46b admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:16.893028 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d11506e-458e-49ea-9d25-fdbb638ca46b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:16.896054 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d11506e-458e-49ea-9d25-fdbb638ca46b admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:17.077142 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1182/2349] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:15:16 2022] POST /identity/v3/projects => generated 363 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:15:17.077662 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2207c3dd-7186-4afb-864a-de1e77dd00cf None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:17.110999 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2207c3dd-7186-4afb-864a-de1e77dd00cf None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:17.232592 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1981769-fc0f-46c4-8e41-d2d30e0f952b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:17.234378 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1981769-fc0f-46c4-8e41-d2d30e0f952b admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:17.234886 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1981769-fc0f-46c4-8e41-d2d30e0f952b admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:17.235683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1981769-fc0f-46c4-8e41-d2d30e0f952b admin admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:17.239455 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1981769-fc0f-46c4-8e41-d2d30e0f952b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:17.243972 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1981769-fc0f-46c4-8e41-d2d30e0f952b admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:17.280823 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65730bfb-a175-44ae-8302-8d47f6ce4f35 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:17.281265 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65730bfb-a175-44ae-8302-8d47f6ce4f35 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:17.281577 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65730bfb-a175-44ae-8302-8d47f6ce4f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:17.295748 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:15:17.312044 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-65730bfb-a175-44ae-8302-8d47f6ce4f35 None None] MFA Rules not processed for user `ecdea0572cc64ca394cd4d977f536d5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:15:17.313624 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-65730bfb-a175-44ae-8302-8d47f6ce4f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:17.385388 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1168/2350] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:15:17 2022] POST /identity/v3/users => generated 288 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 17:15:17.387689 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb0522c-9e0a-4ec4-9194-9cbdde09350f None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:17.390572 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb0522c-9e0a-4ec4-9194-9cbdde09350f None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:17.390572 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb0522c-9e0a-4ec4-9194-9cbdde09350f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:17.438020 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1183/2351] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:15:17 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:15:17.439165 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65730bfb-a175-44ae-8302-8d47f6ce4f35 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:17.442378 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-0bb0522c-9e0a-4ec4-9194-9cbdde09350f None None] MFA Rules not processed for user `284490df3764411c9aa15edbf78b1541`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:15:17.443761 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-0bb0522c-9e0a-4ec4-9194-9cbdde09350f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:17.452376 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65730bfb-a175-44ae-8302-8d47f6ce4f35 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:17.459167 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba179de-0db4-4636-b1f0-e23fa98457a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:17.459906 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba179de-0db4-4636-b1f0-e23fa98457a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:17.460303 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba179de-0db4-4636-b1f0-e23fa98457a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:17.460643 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba179de-0db4-4636-b1f0-e23fa98457a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:17.461358 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba179de-0db4-4636-b1f0-e23fa98457a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:17.463555 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba179de-0db4-4636-b1f0-e23fa98457a5 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:17.478801 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1184/2352] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:15:17 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:15:17.483063 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba179de-0db4-4636-b1f0-e23fa98457a5 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:17.495318 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba179de-0db4-4636-b1f0-e23fa98457a5 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:17.502080 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4007c101-cd6e-471a-b9c2-6bd9041b43b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:17.502755 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4007c101-cd6e-471a-b9c2-6bd9041b43b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:17.503140 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4007c101-cd6e-471a-b9c2-6bd9041b43b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:17.503415 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4007c101-cd6e-471a-b9c2-6bd9041b43b1 admin admin] PATH_INFO: `/v3/projects/5cd69143f8d04b1586760937e9090bd8/users/05e8fb06897942ebb96da967d37c52ac/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:17.531941 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1169/2353] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:15:17 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:15:17.535709 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4007c101-cd6e-471a-b9c2-6bd9041b43b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cd69143f8d04b1586760937e9090bd8, user_id=05e8fb06897942ebb96da967d37c52ac, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:17.539073 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4007c101-cd6e-471a-b9c2-6bd9041b43b1 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:17.552068 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb0522c-9e0a-4ec4-9194-9cbdde09350f None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:17.553760 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-0bb0522c-9e0a-4ec4-9194-9cbdde09350f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:17.639463 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1185/2354] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:15:17 2022] PUT /identity/v3/projects/5cd69143f8d04b1586760937e9090bd8/users/05e8fb06897942ebb96da967d37c52ac/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:15:17.645287 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4007c101-cd6e-471a-b9c2-6bd9041b43b1 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:17.671738 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb0522c-9e0a-4ec4-9194-9cbdde09350f None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:17.742563 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4007c101-cd6e-471a-b9c2-6bd9041b43b1 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:17.755027 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5f4367-f0f3-442b-bd0f-c3a33b4d459f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:17.755778 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f4367-f0f3-442b-bd0f-c3a33b4d459f admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:17.756133 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f4367-f0f3-442b-bd0f-c3a33b4d459f admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:17.756472 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f4367-f0f3-442b-bd0f-c3a33b4d459f admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:17.758998 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5f4367-f0f3-442b-bd0f-c3a33b4d459f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:17.759747 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5f4367-f0f3-442b-bd0f-c3a33b4d459f admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:17.775687 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943520c2-6acb-4fdb-9081-d6e280b42d1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:17.776669 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943520c2-6acb-4fdb-9081-d6e280b42d1e service cinder] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:17.777152 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943520c2-6acb-4fdb-9081-d6e280b42d1e service cinder] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:17.777635 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943520c2-6acb-4fdb-9081-d6e280b42d1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:17.789640 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1186/2355] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:15:17 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 17:15:17.789640 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5f4367-f0f3-442b-bd0f-c3a33b4d459f admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:17.790093 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-943520c2-6acb-4fdb-9081-d6e280b42d1e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:17.802532 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5f4367-f0f3-442b-bd0f-c3a33b4d459f admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:17.821999 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc55429-b4ef-48da-ab2b-bccf28293c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:17.824123 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc55429-b4ef-48da-ab2b-bccf28293c27 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:17.824387 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc55429-b4ef-48da-ab2b-bccf28293c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:17.825277 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc55429-b4ef-48da-ab2b-bccf28293c27 admin admin] PATH_INFO: `/v3/projects/5cd69143f8d04b1586760937e9090bd8/users/05e8fb06897942ebb96da967d37c52ac/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:17.828608 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc55429-b4ef-48da-ab2b-bccf28293c27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cd69143f8d04b1586760937e9090bd8, user_id=05e8fb06897942ebb96da967d37c52ac, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:17.838739 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc55429-b4ef-48da-ab2b-bccf28293c27 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:17.872133 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1187/2356] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:15:17 2022] PUT /identity/v3/projects/5cd69143f8d04b1586760937e9090bd8/users/05e8fb06897942ebb96da967d37c52ac/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:15:17.882853 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b70add4-9316-4805-b97a-0480bd670eb6 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:17.883467 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b70add4-9316-4805-b97a-0480bd670eb6 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:17.883843 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b70add4-9316-4805-b97a-0480bd670eb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:17.923201 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943520c2-6acb-4fdb-9081-d6e280b42d1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:17.924960 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943520c2-6acb-4fdb-9081-d6e280b42d1e service cinder] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:17.940609 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-8b70add4-9316-4805-b97a-0480bd670eb6 None None] MFA Rules not processed for user `05e8fb06897942ebb96da967d37c52ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:15:17.942464 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-8b70add4-9316-4805-b97a-0480bd670eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:17.945250 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1170/2357] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:15:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:15:18.077807 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1188/2358] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:15:17 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:15:18.090743 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943520c2-6acb-4fdb-9081-d6e280b42d1e service cinder] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:18.098246 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-943520c2-6acb-4fdb-9081-d6e280b42d1e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:18.110818 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943520c2-6acb-4fdb-9081-d6e280b42d1e service cinder] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:18.222312 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63da2cec-2852-4b58-acc4-d9260c6991da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:18.223665 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da2cec-2852-4b58-acc4-d9260c6991da service cinder] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:18.224033 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da2cec-2852-4b58-acc4-d9260c6991da service cinder] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:18.224221 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da2cec-2852-4b58-acc4-d9260c6991da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:18.245050 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63da2cec-2852-4b58-acc4-d9260c6991da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:18.246547 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63da2cec-2852-4b58-acc4-d9260c6991da service cinder] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:18.273798 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1171/2359] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:15:18 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:15:18.367911 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:15:18.574890 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b70add4-9316-4805-b97a-0480bd670eb6 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:18.588020 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b70add4-9316-4805-b97a-0480bd670eb6 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:18.599837 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bace02-8663-415f-a453-fea92f416a97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:18.600775 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bace02-8663-415f-a453-fea92f416a97 service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:18.601213 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bace02-8663-415f-a453-fea92f416a97 service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:18.601558 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bace02-8663-415f-a453-fea92f416a97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:18.686972 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bace02-8663-415f-a453-fea92f416a97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:18.692039 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bace02-8663-415f-a453-fea92f416a97 service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:18.702981 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1189/2360] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:15:18 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:15:18.823857 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:15:20.942782 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e04f0a2-399c-41c0-b78d-93606b88dfbd None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:20.943302 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e04f0a2-399c-41c0-b78d-93606b88dfbd None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:20.943407 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e04f0a2-399c-41c0-b78d-93606b88dfbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:21.032124 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-6e04f0a2-399c-41c0-b78d-93606b88dfbd None None] MFA Rules not processed for user `6af30f5ad30b433983e90592fd3f4d8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:15:21.033394 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-6e04f0a2-399c-41c0-b78d-93606b88dfbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:21.168152 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bace02-8663-415f-a453-fea92f416a97 service cinder] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:21.179580 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bace02-8663-415f-a453-fea92f416a97 service cinder] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:21.185171 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d653b02-e0b6-49aa-9c07-0d2c32d58d5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:21.186039 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d653b02-e0b6-49aa-9c07-0d2c32d58d5b service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:21.186468 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d653b02-e0b6-49aa-9c07-0d2c32d58d5b service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:21.186894 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d653b02-e0b6-49aa-9c07-0d2c32d58d5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:21.217201 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1172/2361] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:15:20 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:15:21.232352 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e04f0a2-399c-41c0-b78d-93606b88dfbd None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:21.235784 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-6e04f0a2-399c-41c0-b78d-93606b88dfbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:21.339242 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d653b02-e0b6-49aa-9c07-0d2c32d58d5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:21.340825 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d653b02-e0b6-49aa-9c07-0d2c32d58d5b service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:21.353204 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1190/2362] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:15:21 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 17:15:21.376677 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e04f0a2-399c-41c0-b78d-93606b88dfbd None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:21.383260 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c8c57-74f6-4058-9b76-253275b29a7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55723c756cae4e7ab636054db48de02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:21.386164 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c8c57-74f6-4058-9b76-253275b29a7d service nova] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:21.386164 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c8c57-74f6-4058-9b76-253275b29a7d service nova] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:21.386164 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c8c57-74f6-4058-9b76-253275b29a7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:21.387739 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-eb0c8c57-74f6-4058-9b76-253275b29a7d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:21.399437 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:15:21.464492 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0c8c57-74f6-4058-9b76-253275b29a7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:21.466292 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0c8c57-74f6-4058-9b76-253275b29a7d service nova] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:21.479141 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1173/2363] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:15:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:15:22.622071 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d653b02-e0b6-49aa-9c07-0d2c32d58d5b service cinder] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:22.646623 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:15:22.711316 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d653b02-e0b6-49aa-9c07-0d2c32d58d5b service cinder] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:22.716967 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dace7760-a3c9-453e-bdd3-408d595544ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:22.717721 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace7760-a3c9-453e-bdd3-408d595544ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:22.718071 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace7760-a3c9-453e-bdd3-408d595544ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:22.718414 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace7760-a3c9-453e-bdd3-408d595544ed admin admin] PATH_INFO: `/v3/users/9cb29abd135642fea4d19496d4a12499` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:22.719896 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dace7760-a3c9-453e-bdd3-408d595544ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cb29abd135642fea4d19496d4a12499)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:22.723127 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dace7760-a3c9-453e-bdd3-408d595544ed admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:22.974064 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-dace7760-a3c9-453e-bdd3-408d595544ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cb29abd135642fea4d19496d4a12499'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:23.032663 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-dace7760-a3c9-453e-bdd3-408d595544ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cb29abd135642fea4d19496d4a12499'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:23.033390 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-dace7760-a3c9-453e-bdd3-408d595544ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cb29abd135642fea4d19496d4a12499'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:23.034317 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-dace7760-a3c9-453e-bdd3-408d595544ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cb29abd135642fea4d19496d4a12499'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:23.044854 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-dace7760-a3c9-453e-bdd3-408d595544ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cb29abd135642fea4d19496d4a12499'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:23.063463 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1191/2364] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:15:22 2022] DELETE /identity/v3/users/9cb29abd135642fea4d19496d4a12499 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:15:23.069420 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240ed6e-c92f-4a09-932c-9f9aac33d1e6 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:23.069420 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240ed6e-c92f-4a09-932c-9f9aac33d1e6 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:23.069906 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240ed6e-c92f-4a09-932c-9f9aac33d1e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:23.096296 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-6240ed6e-c92f-4a09-932c-9f9aac33d1e6 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:15:23.097916 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-6240ed6e-c92f-4a09-932c-9f9aac33d1e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:23.182715 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1174/2365] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:15:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 17:15:23.205313 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dace7760-a3c9-453e-bdd3-408d595544ed admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:23.213276 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-dace7760-a3c9-453e-bdd3-408d595544ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:23.282769 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dace7760-a3c9-453e-bdd3-408d595544ed admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:23.287871 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e676adb5-a1cf-4089-beeb-6274f9759789 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:23.288606 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e676adb5-a1cf-4089-beeb-6274f9759789 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:23.288956 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e676adb5-a1cf-4089-beeb-6274f9759789 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:23.289298 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e676adb5-a1cf-4089-beeb-6274f9759789 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:23.302778 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e676adb5-a1cf-4089-beeb-6274f9759789 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:23.304336 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e676adb5-a1cf-4089-beeb-6274f9759789 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:23.314709 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1192/2366] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:15:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 17:15:23.896328 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6240ed6e-c92f-4a09-932c-9f9aac33d1e6 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:23.898030 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-6240ed6e-c92f-4a09-932c-9f9aac33d1e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:23.923771 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6240ed6e-c92f-4a09-932c-9f9aac33d1e6 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:23.931745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a0f200-ec40-4849-899c-094fce5e31d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:23.932930 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a0f200-ec40-4849-899c-094fce5e31d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:23.933570 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a0f200-ec40-4849-899c-094fce5e31d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:23.934071 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a0f200-ec40-4849-899c-094fce5e31d6 admin admin] PATH_INFO: `/v3/projects/68cc38c90f6641adb3c815f20f18f6f7` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:23.936221 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a0f200-ec40-4849-899c-094fce5e31d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68cc38c90f6641adb3c815f20f18f6f7)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:23.938809 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a0f200-ec40-4849-899c-094fce5e31d6 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:24.137494 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-b0a0f200-ec40-4849-899c-094fce5e31d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68cc38c90f6641adb3c815f20f18f6f7', 'request_id': 'req-b0a0f200-ec40-4849-899c-094fce5e31d6'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:24.259038 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-b0a0f200-ec40-4849-899c-094fce5e31d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68cc38c90f6641adb3c815f20f18f6f7', 'request_id': 'req-b0a0f200-ec40-4849-899c-094fce5e31d6'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:24.277002 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-b0a0f200-ec40-4849-899c-094fce5e31d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68cc38c90f6641adb3c815f20f18f6f7', 'request_id': 'req-b0a0f200-ec40-4849-899c-094fce5e31d6'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:24.293122 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-b0a0f200-ec40-4849-899c-094fce5e31d6 admin admin] The token cache is being invalidate because project 68cc38c90f6641adb3c815f20f18f6f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81650) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:15:24.293415 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-b0a0f200-ec40-4849-899c-094fce5e31d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:24.295666 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-b0a0f200-ec40-4849-899c-094fce5e31d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:24.297011 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1175/2367] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:15:23 2022] DELETE /identity/v3/projects/68cc38c90f6641adb3c815f20f18f6f7 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:15:24.301382 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e676adb5-a1cf-4089-beeb-6274f9759789 service neutron] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:24.314045 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e676adb5-a1cf-4089-beeb-6274f9759789 service neutron] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:24.368143 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:15:24.393009 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02bedd8-45d1-41ce-a7c8-e322636de856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:24.394388 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02bedd8-45d1-41ce-a7c8-e322636de856 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:24.395058 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02bedd8-45d1-41ce-a7c8-e322636de856 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:24.395575 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02bedd8-45d1-41ce-a7c8-e322636de856 admin admin] PATH_INFO: `/v3/users/05e8fb06897942ebb96da967d37c52ac` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:24.399865 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02bedd8-45d1-41ce-a7c8-e322636de856 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05e8fb06897942ebb96da967d37c52ac)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:24.399865 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02bedd8-45d1-41ce-a7c8-e322636de856 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:24.665152 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e02bedd8-45d1-41ce-a7c8-e322636de856 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05e8fb06897942ebb96da967d37c52ac'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:24.812385 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e02bedd8-45d1-41ce-a7c8-e322636de856 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05e8fb06897942ebb96da967d37c52ac'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:24.813275 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e02bedd8-45d1-41ce-a7c8-e322636de856 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05e8fb06897942ebb96da967d37c52ac'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:24.813902 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e02bedd8-45d1-41ce-a7c8-e322636de856 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05e8fb06897942ebb96da967d37c52ac'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:24.822471 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e02bedd8-45d1-41ce-a7c8-e322636de856 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05e8fb06897942ebb96da967d37c52ac'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:24.841354 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1193/2368] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:15:24 2022] DELETE /identity/v3/users/05e8fb06897942ebb96da967d37c52ac => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:15:25.021261 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b01f19-c981-4ae6-bb68-8e4897b6353f None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:25.021551 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b01f19-c981-4ae6-bb68-8e4897b6353f None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:25.022057 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b01f19-c981-4ae6-bb68-8e4897b6353f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:25.043243 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-c5b01f19-c981-4ae6-bb68-8e4897b6353f None None] MFA Rules not processed for user `ebc5df40911b4186abec99c01e0702a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:15:25.052356 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-c5b01f19-c981-4ae6-bb68-8e4897b6353f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:25.086500 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:15:25.122689 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1176/2369] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:15:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:15:25.527734 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02bedd8-45d1-41ce-a7c8-e322636de856 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:25.529881 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-e02bedd8-45d1-41ce-a7c8-e322636de856 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:25.625188 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02bedd8-45d1-41ce-a7c8-e322636de856 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:25.630816 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f2c9f2-1baa-4bed-b2c0-1378b28d3539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:25.631577 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f2c9f2-1baa-4bed-b2c0-1378b28d3539 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:25.631930 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f2c9f2-1baa-4bed-b2c0-1378b28d3539 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:25.632305 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f2c9f2-1baa-4bed-b2c0-1378b28d3539 admin admin] PATH_INFO: `/v3/projects/5cd69143f8d04b1586760937e9090bd8` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:25.633656 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f2c9f2-1baa-4bed-b2c0-1378b28d3539 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cd69143f8d04b1586760937e9090bd8)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:25.635732 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f2c9f2-1baa-4bed-b2c0-1378b28d3539 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:25.823771 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-f0f2c9f2-1baa-4bed-b2c0-1378b28d3539 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cd69143f8d04b1586760937e9090bd8', 'request_id': 'req-f0f2c9f2-1baa-4bed-b2c0-1378b28d3539'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:25.834764 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-f0f2c9f2-1baa-4bed-b2c0-1378b28d3539 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cd69143f8d04b1586760937e9090bd8', 'request_id': 'req-f0f2c9f2-1baa-4bed-b2c0-1378b28d3539'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:25.940518 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-f0f2c9f2-1baa-4bed-b2c0-1378b28d3539 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cd69143f8d04b1586760937e9090bd8', 'request_id': 'req-f0f2c9f2-1baa-4bed-b2c0-1378b28d3539'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:25.953014 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-f0f2c9f2-1baa-4bed-b2c0-1378b28d3539 admin admin] The token cache is being invalidate because project 5cd69143f8d04b1586760937e9090bd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:15:25.953309 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-f0f2c9f2-1baa-4bed-b2c0-1378b28d3539 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:25.953963 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-f0f2c9f2-1baa-4bed-b2c0-1378b28d3539 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:25.955535 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1194/2370] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:15:25 2022] DELETE /identity/v3/projects/5cd69143f8d04b1586760937e9090bd8 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:15:25.960439 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d27c0ce-762c-4a97-b535-d781d198cde5 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:25.960905 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d27c0ce-762c-4a97-b535-d781d198cde5 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:25.960990 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d27c0ce-762c-4a97-b535-d781d198cde5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:25.990528 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-0d27c0ce-762c-4a97-b535-d781d198cde5 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:15:25.991776 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-0d27c0ce-762c-4a97-b535-d781d198cde5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:26.062120 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1177/2371] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:15:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:15:26.070399 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1cd36-187b-4e8f-b6a2-d7061d3a7893 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:26.070690 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1cd36-187b-4e8f-b6a2-d7061d3a7893 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:26.071002 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1cd36-187b-4e8f-b6a2-d7061d3a7893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:26.091379 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-bde1cd36-187b-4e8f-b6a2-d7061d3a7893 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:15:26.092600 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-bde1cd36-187b-4e8f-b6a2-d7061d3a7893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:26.100984 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1195/2372] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:15:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:15:26.105744 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d27c0ce-762c-4a97-b535-d781d198cde5 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:26.116131 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d27c0ce-762c-4a97-b535-d781d198cde5 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:26.120967 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4cad25-1f76-438f-91ac-bc1e255163a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:26.121582 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4cad25-1f76-438f-91ac-bc1e255163a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:26.121928 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4cad25-1f76-438f-91ac-bc1e255163a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:26.122206 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4cad25-1f76-438f-91ac-bc1e255163a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:26.122890 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4cad25-1f76-438f-91ac-bc1e255163a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:15:26.123220 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4cad25-1f76-438f-91ac-bc1e255163a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:26.124963 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4cad25-1f76-438f-91ac-bc1e255163a0 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:26.141192 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1178/2373] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:15:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 17:15:26.142334 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde1cd36-187b-4e8f-b6a2-d7061d3a7893 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:26.169897 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde1cd36-187b-4e8f-b6a2-d7061d3a7893 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:26.169897 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3a38d4-4212-4aa4-b535-270f7cc6cd00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:26.169897 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3a38d4-4212-4aa4-b535-270f7cc6cd00 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:26.169897 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3a38d4-4212-4aa4-b535-270f7cc6cd00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:26.170228 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3a38d4-4212-4aa4-b535-270f7cc6cd00 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:26.170228 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3a38d4-4212-4aa4-b535-270f7cc6cd00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:26.170670 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3a38d4-4212-4aa4-b535-270f7cc6cd00 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:26.265779 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1196/2374] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:15:26 2022] POST /identity/v3/projects => generated 381 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 17:15:26.270046 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4cad25-1f76-438f-91ac-bc1e255163a0 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:26.281237 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4cad25-1f76-438f-91ac-bc1e255163a0 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:26.341396 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c997d9-0110-491a-8434-7042ecea8655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:26.342232 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c997d9-0110-491a-8434-7042ecea8655 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:26.342552 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c997d9-0110-491a-8434-7042ecea8655 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:26.342780 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c997d9-0110-491a-8434-7042ecea8655 admin admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:26.343435 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c997d9-0110-491a-8434-7042ecea8655 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:26.345361 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c997d9-0110-491a-8434-7042ecea8655 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:26.448204 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1179/2375] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:15:26 2022] POST /identity/v3/users => generated 291 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:15:26.453352 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3a38d4-4212-4aa4-b535-270f7cc6cd00 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:26.465507 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3a38d4-4212-4aa4-b535-270f7cc6cd00 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:26.470360 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2266d3-ae9d-4128-8679-331016bb6603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:26.471038 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2266d3-ae9d-4128-8679-331016bb6603 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:26.471346 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2266d3-ae9d-4128-8679-331016bb6603 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:26.477613 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2266d3-ae9d-4128-8679-331016bb6603 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:26.477613 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2266d3-ae9d-4128-8679-331016bb6603 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:26.477613 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2266d3-ae9d-4128-8679-331016bb6603 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:26.485068 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1197/2376] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:15:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:15:26.489231 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c997d9-0110-491a-8434-7042ecea8655 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:26.501239 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c997d9-0110-491a-8434-7042ecea8655 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:26.509285 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbaeb78-4772-416e-bcd1-da9273b9ffc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:26.510520 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbaeb78-4772-416e-bcd1-da9273b9ffc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:26.510799 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbaeb78-4772-416e-bcd1-da9273b9ffc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:26.511297 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbaeb78-4772-416e-bcd1-da9273b9ffc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:26.511940 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbaeb78-4772-416e-bcd1-da9273b9ffc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:26.513676 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbaeb78-4772-416e-bcd1-da9273b9ffc9 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:26.529503 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1180/2377] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:15:26 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:15:26.530865 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2266d3-ae9d-4128-8679-331016bb6603 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:26.541344 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2266d3-ae9d-4128-8679-331016bb6603 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:26.546037 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6fd257-625f-4f03-bf90-1f912f0a152d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:26.546682 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6fd257-625f-4f03-bf90-1f912f0a152d admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:26.546995 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6fd257-625f-4f03-bf90-1f912f0a152d admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:26.547447 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6fd257-625f-4f03-bf90-1f912f0a152d admin admin] PATH_INFO: `/v3/projects/1e554f3c897c44239405fec74358d261/users/6607d71e709a4b34a90268f1ac4f30a3/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:26.564663 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6fd257-625f-4f03-bf90-1f912f0a152d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e554f3c897c44239405fec74358d261, user_id=6607d71e709a4b34a90268f1ac4f30a3, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:26.568239 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6fd257-625f-4f03-bf90-1f912f0a152d admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:26.694139 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1198/2378] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:15:26 2022] PUT /identity/v3/projects/1e554f3c897c44239405fec74358d261/users/6607d71e709a4b34a90268f1ac4f30a3/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:15:26.710232 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1af5c3-5fa4-473e-b3e8-3d3c67d6a358 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:26.710632 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1af5c3-5fa4-473e-b3e8-3d3c67d6a358 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:26.711013 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1af5c3-5fa4-473e-b3e8-3d3c67d6a358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:26.743435 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-5d1af5c3-5fa4-473e-b3e8-3d3c67d6a358 None None] MFA Rules not processed for user `6607d71e709a4b34a90268f1ac4f30a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:15:26.744667 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-5d1af5c3-5fa4-473e-b3e8-3d3c67d6a358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:26.818328 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1181/2379] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:15:26 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:15:26.825595 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6fd257-625f-4f03-bf90-1f912f0a152d admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:26.891367 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6fd257-625f-4f03-bf90-1f912f0a152d admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:26.896495 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef10da-4102-4a5e-8492-11c74d752f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:26.897141 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef10da-4102-4a5e-8492-11c74d752f9d admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:26.897428 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef10da-4102-4a5e-8492-11c74d752f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:26.900968 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef10da-4102-4a5e-8492-11c74d752f9d admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:26.900968 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bef10da-4102-4a5e-8492-11c74d752f9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:26.900968 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bef10da-4102-4a5e-8492-11c74d752f9d admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:26.923597 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:15:27.060145 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1199/2380] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:15:26 2022] POST /identity/v3/projects => generated 381 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:15:27.060576 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1af5c3-5fa4-473e-b3e8-3d3c67d6a358 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:27.060576 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1af5c3-5fa4-473e-b3e8-3d3c67d6a358 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:27.109788 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947eb015-3f95-4626-8ae7-0a8a9a99cc7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:27.110421 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947eb015-3f95-4626-8ae7-0a8a9a99cc7e admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:27.110703 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947eb015-3f95-4626-8ae7-0a8a9a99cc7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:27.111011 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947eb015-3f95-4626-8ae7-0a8a9a99cc7e admin admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:27.111636 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947eb015-3f95-4626-8ae7-0a8a9a99cc7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:27.114631 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947eb015-3f95-4626-8ae7-0a8a9a99cc7e admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:27.639549 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1182/2381] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:15:27 2022] POST /identity/v3/users => generated 297 bytes in 595 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 17:15:27.642499 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef10da-4102-4a5e-8492-11c74d752f9d admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:27.653897 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef10da-4102-4a5e-8492-11c74d752f9d admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:27.658651 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabe6195-e75f-4d38-b503-c7550b54a22c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:27.659306 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe6195-e75f-4d38-b503-c7550b54a22c admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:27.664515 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe6195-e75f-4d38-b503-c7550b54a22c admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:27.664515 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe6195-e75f-4d38-b503-c7550b54a22c admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:27.664515 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabe6195-e75f-4d38-b503-c7550b54a22c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:27.664515 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabe6195-e75f-4d38-b503-c7550b54a22c admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:27.673906 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1200/2382] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:15:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:15:27.678644 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947eb015-3f95-4626-8ae7-0a8a9a99cc7e admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:27.688984 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947eb015-3f95-4626-8ae7-0a8a9a99cc7e admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:27.695207 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49953fbc-3145-4e21-abf0-c2141285654a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:27.695871 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49953fbc-3145-4e21-abf0-c2141285654a admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:27.696170 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49953fbc-3145-4e21-abf0-c2141285654a admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:27.696445 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49953fbc-3145-4e21-abf0-c2141285654a admin admin] PATH_INFO: `/v3/projects/b46158ac66e34254a3e66485cce93da9/users/4c46f830a1564581b6a08767502b2009/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:27.711802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49953fbc-3145-4e21-abf0-c2141285654a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b46158ac66e34254a3e66485cce93da9, user_id=4c46f830a1564581b6a08767502b2009, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:27.714509 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49953fbc-3145-4e21-abf0-c2141285654a admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:27.781330 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1183/2383] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:15:27 2022] PUT /identity/v3/projects/b46158ac66e34254a3e66485cce93da9/users/4c46f830a1564581b6a08767502b2009/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:15:27.785660 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabe6195-e75f-4d38-b503-c7550b54a22c admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:27.796189 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabe6195-e75f-4d38-b503-c7550b54a22c admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:27.888098 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862c5c9c-61df-4e6f-bc48-bd5f5f0c3bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:27.888835 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862c5c9c-61df-4e6f-bc48-bd5f5f0c3bcb admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:27.889217 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862c5c9c-61df-4e6f-bc48-bd5f5f0c3bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:27.889640 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862c5c9c-61df-4e6f-bc48-bd5f5f0c3bcb admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:27.890692 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862c5c9c-61df-4e6f-bc48-bd5f5f0c3bcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:27.892448 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862c5c9c-61df-4e6f-bc48-bd5f5f0c3bcb admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:27.912491 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1201/2384] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:15:27 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:15:27.918651 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49953fbc-3145-4e21-abf0-c2141285654a admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:27.928283 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49953fbc-3145-4e21-abf0-c2141285654a admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:27.939384 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986e4abf-209d-4993-a180-04611f9979f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:27.940208 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e4abf-209d-4993-a180-04611f9979f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:27.940619 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e4abf-209d-4993-a180-04611f9979f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:27.941023 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e4abf-209d-4993-a180-04611f9979f1 admin admin] PATH_INFO: `/v3/projects/b46158ac66e34254a3e66485cce93da9/users/4c46f830a1564581b6a08767502b2009/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:27.943787 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986e4abf-209d-4993-a180-04611f9979f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b46158ac66e34254a3e66485cce93da9, user_id=4c46f830a1564581b6a08767502b2009, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:27.946745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986e4abf-209d-4993-a180-04611f9979f1 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:27.959602 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1184/2385] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:15:27 2022] PUT /identity/v3/projects/b46158ac66e34254a3e66485cce93da9/users/4c46f830a1564581b6a08767502b2009/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:15:27.972471 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:15:27.975745 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60185f0b-0b2e-408a-9731-4d9971c85d31 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:27.976274 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60185f0b-0b2e-408a-9731-4d9971c85d31 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:27.976857 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60185f0b-0b2e-408a-9731-4d9971c85d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:28.013495 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-60185f0b-0b2e-408a-9731-4d9971c85d31 None None] MFA Rules not processed for user `4c46f830a1564581b6a08767502b2009`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:15:28.014835 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-60185f0b-0b2e-408a-9731-4d9971c85d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:28.119628 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1202/2386] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:15:27 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:15:28.127555 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986e4abf-209d-4993-a180-04611f9979f1 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:28.130476 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-986e4abf-209d-4993-a180-04611f9979f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:28.207969 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986e4abf-209d-4993-a180-04611f9979f1 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:28.211834 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc89700-98d8-4e76-9575-b8a04ccb8e33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:28.216103 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc89700-98d8-4e76-9575-b8a04ccb8e33 service cinder] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:28.216103 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc89700-98d8-4e76-9575-b8a04ccb8e33 service cinder] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:28.216103 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc89700-98d8-4e76-9575-b8a04ccb8e33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:28.222968 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc89700-98d8-4e76-9575-b8a04ccb8e33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:28.224435 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc89700-98d8-4e76-9575-b8a04ccb8e33 service cinder] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:28.234658 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1185/2387] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:15:28 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:15:28.755034 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60185f0b-0b2e-408a-9731-4d9971c85d31 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:28.812169 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:15:28.822944 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60185f0b-0b2e-408a-9731-4d9971c85d31 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:28.827479 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d93d1b-1e5f-4bed-affd-14bcd73d95cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:28.828177 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d93d1b-1e5f-4bed-affd-14bcd73d95cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:28.828460 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d93d1b-1e5f-4bed-affd-14bcd73d95cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:28.828738 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d93d1b-1e5f-4bed-affd-14bcd73d95cb admin admin] PATH_INFO: `/v3/users/6607d71e709a4b34a90268f1ac4f30a3` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:28.829852 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d93d1b-1e5f-4bed-affd-14bcd73d95cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=6607d71e709a4b34a90268f1ac4f30a3)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:28.833270 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d93d1b-1e5f-4bed-affd-14bcd73d95cb admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:28.968964 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-98d93d1b-1e5f-4bed-affd-14bcd73d95cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6607d71e709a4b34a90268f1ac4f30a3'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:29.023380 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-98d93d1b-1e5f-4bed-affd-14bcd73d95cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6607d71e709a4b34a90268f1ac4f30a3'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:29.024130 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-98d93d1b-1e5f-4bed-affd-14bcd73d95cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6607d71e709a4b34a90268f1ac4f30a3'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:29.024825 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-98d93d1b-1e5f-4bed-affd-14bcd73d95cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6607d71e709a4b34a90268f1ac4f30a3'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:29.032581 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-98d93d1b-1e5f-4bed-affd-14bcd73d95cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6607d71e709a4b34a90268f1ac4f30a3'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:29.052070 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1203/2388] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:15:28 2022] DELETE /identity/v3/users/6607d71e709a4b34a90268f1ac4f30a3 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:15:29.065311 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52367ed-50c4-4e97-82ee-f0b9ad5c991a None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:29.065603 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52367ed-50c4-4e97-82ee-f0b9ad5c991a None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:29.065883 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52367ed-50c4-4e97-82ee-f0b9ad5c991a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:29.086356 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-b52367ed-50c4-4e97-82ee-f0b9ad5c991a None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:15:29.087602 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-b52367ed-50c4-4e97-82ee-f0b9ad5c991a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:29.112494 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:15:29.148763 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1186/2389] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:15:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:15:29.160070 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d93d1b-1e5f-4bed-affd-14bcd73d95cb admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:29.162360 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-98d93d1b-1e5f-4bed-affd-14bcd73d95cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:29.257032 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d93d1b-1e5f-4bed-affd-14bcd73d95cb admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:29.260678 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a49ee7-a49b-4cb5-8145-8810d11a1487 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:29.261406 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a49ee7-a49b-4cb5-8145-8810d11a1487 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:29.261683 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a49ee7-a49b-4cb5-8145-8810d11a1487 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:29.262074 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a49ee7-a49b-4cb5-8145-8810d11a1487 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:29.273360 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a49ee7-a49b-4cb5-8145-8810d11a1487 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:29.274932 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a49ee7-a49b-4cb5-8145-8810d11a1487 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:29.286036 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1204/2390] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:15:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:15:29.765650 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52367ed-50c4-4e97-82ee-f0b9ad5c991a None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:29.767600 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-b52367ed-50c4-4e97-82ee-f0b9ad5c991a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:29.777771 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:15:29.781281 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52367ed-50c4-4e97-82ee-f0b9ad5c991a None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:29.790454 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab0c9ab-be09-426d-9e04-9bfe08f3b0b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:29.791324 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab0c9ab-be09-426d-9e04-9bfe08f3b0b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:29.791708 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab0c9ab-be09-426d-9e04-9bfe08f3b0b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:29.792194 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab0c9ab-be09-426d-9e04-9bfe08f3b0b6 admin admin] PATH_INFO: `/v3/projects/1e554f3c897c44239405fec74358d261` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:29.793407 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab0c9ab-be09-426d-9e04-9bfe08f3b0b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e554f3c897c44239405fec74358d261)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:29.795626 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab0c9ab-be09-426d-9e04-9bfe08f3b0b6 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:29.921283 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-8ab0c9ab-be09-426d-9e04-9bfe08f3b0b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e554f3c897c44239405fec74358d261', 'request_id': 'req-8ab0c9ab-be09-426d-9e04-9bfe08f3b0b6'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:30.022978 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-8ab0c9ab-be09-426d-9e04-9bfe08f3b0b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e554f3c897c44239405fec74358d261', 'request_id': 'req-8ab0c9ab-be09-426d-9e04-9bfe08f3b0b6'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:30.029424 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-8ab0c9ab-be09-426d-9e04-9bfe08f3b0b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e554f3c897c44239405fec74358d261', 'request_id': 'req-8ab0c9ab-be09-426d-9e04-9bfe08f3b0b6'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:30.041714 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-8ab0c9ab-be09-426d-9e04-9bfe08f3b0b6 admin admin] The token cache is being invalidate because project 1e554f3c897c44239405fec74358d261 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81650) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:15:30.041968 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-8ab0c9ab-be09-426d-9e04-9bfe08f3b0b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:30.042615 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-8ab0c9ab-be09-426d-9e04-9bfe08f3b0b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:30.044468 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1187/2391] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:15:29 2022] DELETE /identity/v3/projects/1e554f3c897c44239405fec74358d261 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:15:30.048363 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a49ee7-a49b-4cb5-8145-8810d11a1487 service neutron] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:30.058370 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a49ee7-a49b-4cb5-8145-8810d11a1487 service neutron] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:30.117690 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfc4293-4691-4035-bd31-bd17d2306de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:30.118477 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfc4293-4691-4035-bd31-bd17d2306de7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:30.118947 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfc4293-4691-4035-bd31-bd17d2306de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:30.119245 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfc4293-4691-4035-bd31-bd17d2306de7 admin admin] PATH_INFO: `/v3/users/4c46f830a1564581b6a08767502b2009` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:30.120644 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfc4293-4691-4035-bd31-bd17d2306de7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c46f830a1564581b6a08767502b2009)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:30.122885 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfc4293-4691-4035-bd31-bd17d2306de7 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:30.294943 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-4cfc4293-4691-4035-bd31-bd17d2306de7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c46f830a1564581b6a08767502b2009'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:30.368967 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-4cfc4293-4691-4035-bd31-bd17d2306de7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c46f830a1564581b6a08767502b2009'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:30.369630 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-4cfc4293-4691-4035-bd31-bd17d2306de7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c46f830a1564581b6a08767502b2009'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:30.370246 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-4cfc4293-4691-4035-bd31-bd17d2306de7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c46f830a1564581b6a08767502b2009'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:30.377830 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-4cfc4293-4691-4035-bd31-bd17d2306de7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c46f830a1564581b6a08767502b2009'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:30.399707 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1205/2392] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:15:30 2022] DELETE /identity/v3/users/4c46f830a1564581b6a08767502b2009 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:15:30.858616 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab0c9ab-be09-426d-9e04-9bfe08f3b0b6 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:30.860334 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-8ab0c9ab-be09-426d-9e04-9bfe08f3b0b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:30.874952 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab0c9ab-be09-426d-9e04-9bfe08f3b0b6 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:30.955819 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8744ffd1-1cb9-42a9-a1a0-804702be985d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:30.955819 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8744ffd1-1cb9-42a9-a1a0-804702be985d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:30.955819 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8744ffd1-1cb9-42a9-a1a0-804702be985d admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:30.955819 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8744ffd1-1cb9-42a9-a1a0-804702be985d admin admin] PATH_INFO: `/v3/projects/b46158ac66e34254a3e66485cce93da9` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:30.956175 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8744ffd1-1cb9-42a9-a1a0-804702be985d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b46158ac66e34254a3e66485cce93da9)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:30.956175 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8744ffd1-1cb9-42a9-a1a0-804702be985d admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:31.089848 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-8744ffd1-1cb9-42a9-a1a0-804702be985d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b46158ac66e34254a3e66485cce93da9', 'request_id': 'req-8744ffd1-1cb9-42a9-a1a0-804702be985d'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:31.093332 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:15:31.163022 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-8744ffd1-1cb9-42a9-a1a0-804702be985d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b46158ac66e34254a3e66485cce93da9', 'request_id': 'req-8744ffd1-1cb9-42a9-a1a0-804702be985d'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:31.169161 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-8744ffd1-1cb9-42a9-a1a0-804702be985d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b46158ac66e34254a3e66485cce93da9', 'request_id': 'req-8744ffd1-1cb9-42a9-a1a0-804702be985d'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:31.180643 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-8744ffd1-1cb9-42a9-a1a0-804702be985d admin admin] The token cache is being invalidate because project b46158ac66e34254a3e66485cce93da9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81650) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:15:31.184745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-8744ffd1-1cb9-42a9-a1a0-804702be985d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:31.184745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-8744ffd1-1cb9-42a9-a1a0-804702be985d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:31.184745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1188/2393] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:15:30 2022] DELETE /identity/v3/projects/b46158ac66e34254a3e66485cce93da9 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:15:31.188175 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee0269-ec2f-4504-8fdf-e8a7d0513d83 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:31.188654 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee0269-ec2f-4504-8fdf-e8a7d0513d83 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:31.188765 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee0269-ec2f-4504-8fdf-e8a7d0513d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:31.211997 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-f3ee0269-ec2f-4504-8fdf-e8a7d0513d83 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:15:31.213268 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-f3ee0269-ec2f-4504-8fdf-e8a7d0513d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:31.225768 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:15:31.273678 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1206/2394] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:15:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:15:31.282517 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85836306-1d09-483a-9b19-97ed7d7c0306 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:31.282809 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85836306-1d09-483a-9b19-97ed7d7c0306 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:31.283122 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85836306-1d09-483a-9b19-97ed7d7c0306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:31.304277 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-85836306-1d09-483a-9b19-97ed7d7c0306 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:15:31.305493 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-85836306-1d09-483a-9b19-97ed7d7c0306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:31.313829 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1189/2395] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:15:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:15:31.318925 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ee0269-ec2f-4504-8fdf-e8a7d0513d83 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:31.328823 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ee0269-ec2f-4504-8fdf-e8a7d0513d83 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:31.334122 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7359bbc-58b8-4f1f-b1e9-6590d3d6ead7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:31.334758 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7359bbc-58b8-4f1f-b1e9-6590d3d6ead7 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:31.335095 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7359bbc-58b8-4f1f-b1e9-6590d3d6ead7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:31.335384 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7359bbc-58b8-4f1f-b1e9-6590d3d6ead7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:31.336022 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7359bbc-58b8-4f1f-b1e9-6590d3d6ead7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:15:31.336329 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7359bbc-58b8-4f1f-b1e9-6590d3d6ead7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:31.337947 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7359bbc-58b8-4f1f-b1e9-6590d3d6ead7 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:31.349719 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1207/2396] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:15:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 17:15:31.354208 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85836306-1d09-483a-9b19-97ed7d7c0306 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:31.364254 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85836306-1d09-483a-9b19-97ed7d7c0306 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:31.368956 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d93b23-b79a-46f8-ad2c-0ada783bf963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:31.369603 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d93b23-b79a-46f8-ad2c-0ada783bf963 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:31.369877 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d93b23-b79a-46f8-ad2c-0ada783bf963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:31.370142 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d93b23-b79a-46f8-ad2c-0ada783bf963 admin admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:31.370753 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d93b23-b79a-46f8-ad2c-0ada783bf963 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:31.372745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d93b23-b79a-46f8-ad2c-0ada783bf963 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:31.463790 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1190/2397] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:15:31 2022] POST /identity/v3/projects => generated 359 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:15:31.468243 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7359bbc-58b8-4f1f-b1e9-6590d3d6ead7 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:31.479826 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7359bbc-58b8-4f1f-b1e9-6590d3d6ead7 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:31.537461 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d6b09b-c379-49ef-8f23-9269ec6316a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:31.538079 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d6b09b-c379-49ef-8f23-9269ec6316a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:31.538363 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d6b09b-c379-49ef-8f23-9269ec6316a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:31.538670 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d6b09b-c379-49ef-8f23-9269ec6316a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:31.539350 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d6b09b-c379-49ef-8f23-9269ec6316a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:31.541519 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d6b09b-c379-49ef-8f23-9269ec6316a8 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:31.636082 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1208/2398] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:15:31 2022] POST /identity/v3/users => generated 280 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:15:31.640052 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d93b23-b79a-46f8-ad2c-0ada783bf963 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:31.650704 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d93b23-b79a-46f8-ad2c-0ada783bf963 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:31.656772 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3579330c-75f6-4d21-bda9-e250474e5381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:31.657061 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3579330c-75f6-4d21-bda9-e250474e5381 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:31.657346 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3579330c-75f6-4d21-bda9-e250474e5381 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:31.657630 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3579330c-75f6-4d21-bda9-e250474e5381 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:31.658282 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3579330c-75f6-4d21-bda9-e250474e5381 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:31.659979 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3579330c-75f6-4d21-bda9-e250474e5381 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:31.674381 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1191/2399] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:15:31.676403 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d6b09b-c379-49ef-8f23-9269ec6316a8 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:31.687651 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d6b09b-c379-49ef-8f23-9269ec6316a8 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:31.693482 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531515a2-cff7-4c8f-93b0-e9457b23b363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:31.694118 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531515a2-cff7-4c8f-93b0-e9457b23b363 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:31.694765 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531515a2-cff7-4c8f-93b0-e9457b23b363 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:31.695147 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531515a2-cff7-4c8f-93b0-e9457b23b363 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:31.695766 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531515a2-cff7-4c8f-93b0-e9457b23b363 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:31.697393 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531515a2-cff7-4c8f-93b0-e9457b23b363 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:31.709639 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1209/2400] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 17:15:31.714991 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3579330c-75f6-4d21-bda9-e250474e5381 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:31.726831 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3579330c-75f6-4d21-bda9-e250474e5381 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:31.732922 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b91ebe-3c1c-4cd9-9c2c-1b4d56af091b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:31.733712 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b91ebe-3c1c-4cd9-9c2c-1b4d56af091b admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:31.734502 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b91ebe-3c1c-4cd9-9c2c-1b4d56af091b admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:31.734502 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b91ebe-3c1c-4cd9-9c2c-1b4d56af091b admin admin] PATH_INFO: `/v3/projects/6163225642c64821abc1d2e30f81400f/users/e94ea0308da8461a9f62f3cd8f5b5103/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:31.767172 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b91ebe-3c1c-4cd9-9c2c-1b4d56af091b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6163225642c64821abc1d2e30f81400f, user_id=e94ea0308da8461a9f62f3cd8f5b5103, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:31.771817 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b91ebe-3c1c-4cd9-9c2c-1b4d56af091b admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:31.912934 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1192/2401] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:15:31 2022] PUT /identity/v3/projects/6163225642c64821abc1d2e30f81400f/users/e94ea0308da8461a9f62f3cd8f5b5103/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:15:31.918949 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a6da6f-1c61-4565-894d-3ef03c62f249 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:31.919242 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a6da6f-1c61-4565-894d-3ef03c62f249 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:31.919515 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a6da6f-1c61-4565-894d-3ef03c62f249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:31.951353 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-a9a6da6f-1c61-4565-894d-3ef03c62f249 None None] MFA Rules not processed for user `e94ea0308da8461a9f62f3cd8f5b5103`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:15:31.953134 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-a9a6da6f-1c61-4565-894d-3ef03c62f249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:32.021485 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1210/2402] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:15:31 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:15:32.060964 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b91ebe-3c1c-4cd9-9c2c-1b4d56af091b admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:32.064167 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-29b91ebe-3c1c-4cd9-9c2c-1b4d56af091b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:32.132630 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b91ebe-3c1c-4cd9-9c2c-1b4d56af091b admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:32.136703 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c6fb88-ecdb-40b2-8583-72dcdb0446f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:32.137323 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c6fb88-ecdb-40b2-8583-72dcdb0446f4 service cinder] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:32.137635 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c6fb88-ecdb-40b2-8583-72dcdb0446f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:32.137888 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c6fb88-ecdb-40b2-8583-72dcdb0446f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:32.148519 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c6fb88-ecdb-40b2-8583-72dcdb0446f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:32.150448 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c6fb88-ecdb-40b2-8583-72dcdb0446f4 service cinder] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:32.162424 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1193/2403] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:15:32 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:15:32.180468 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a6da6f-1c61-4565-894d-3ef03c62f249 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:32.248492 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a6da6f-1c61-4565-894d-3ef03c62f249 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:32.253206 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b1d4ab-4d7b-44ba-8343-f27b4df35511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:32.253886 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b1d4ab-4d7b-44ba-8343-f27b4df35511 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:32.254168 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b1d4ab-4d7b-44ba-8343-f27b4df35511 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:32.254437 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b1d4ab-4d7b-44ba-8343-f27b4df35511 admin admin] PATH_INFO: `/v3/users/e94ea0308da8461a9f62f3cd8f5b5103` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:32.255706 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b1d4ab-4d7b-44ba-8343-f27b4df35511 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e94ea0308da8461a9f62f3cd8f5b5103)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:32.257531 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b1d4ab-4d7b-44ba-8343-f27b4df35511 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:32.266822 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:15:32.445062 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-88b1d4ab-4d7b-44ba-8343-f27b4df35511 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e94ea0308da8461a9f62f3cd8f5b5103'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:32.525777 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-88b1d4ab-4d7b-44ba-8343-f27b4df35511 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e94ea0308da8461a9f62f3cd8f5b5103'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:32.526642 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-88b1d4ab-4d7b-44ba-8343-f27b4df35511 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e94ea0308da8461a9f62f3cd8f5b5103'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:32.527361 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-88b1d4ab-4d7b-44ba-8343-f27b4df35511 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e94ea0308da8461a9f62f3cd8f5b5103'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:32.540537 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-88b1d4ab-4d7b-44ba-8343-f27b4df35511 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e94ea0308da8461a9f62f3cd8f5b5103'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:32.569008 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1211/2404] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:15:32 2022] DELETE /identity/v3/users/e94ea0308da8461a9f62f3cd8f5b5103 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:15:32.573462 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b49e4d-bceb-4811-a8a0-96a9a0332d80 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:32.573856 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b49e4d-bceb-4811-a8a0-96a9a0332d80 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:32.574236 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b49e4d-bceb-4811-a8a0-96a9a0332d80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:32.621437 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-c5b49e4d-bceb-4811-a8a0-96a9a0332d80 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:15:32.623054 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-c5b49e4d-bceb-4811-a8a0-96a9a0332d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:32.701716 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1194/2405] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:15:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:15:32.712894 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b1d4ab-4d7b-44ba-8343-f27b4df35511 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:32.714394 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-88b1d4ab-4d7b-44ba-8343-f27b4df35511 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:32.776017 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b1d4ab-4d7b-44ba-8343-f27b4df35511 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:32.780089 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15aacdf-7013-4421-a94e-5769b9943942 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:32.780720 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15aacdf-7013-4421-a94e-5769b9943942 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:32.781021 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15aacdf-7013-4421-a94e-5769b9943942 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:32.781302 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15aacdf-7013-4421-a94e-5769b9943942 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:32.791675 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15aacdf-7013-4421-a94e-5769b9943942 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:32.793362 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15aacdf-7013-4421-a94e-5769b9943942 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:32.805265 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1212/2406] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:15:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:15:33.235864 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b49e4d-bceb-4811-a8a0-96a9a0332d80 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:33.237452 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-c5b49e4d-bceb-4811-a8a0-96a9a0332d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:33.256249 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b49e4d-bceb-4811-a8a0-96a9a0332d80 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:33.261916 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926e55b9-edd6-4633-8ae3-bb701565bd45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:33.262652 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926e55b9-edd6-4633-8ae3-bb701565bd45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:33.263288 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926e55b9-edd6-4633-8ae3-bb701565bd45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:33.263810 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926e55b9-edd6-4633-8ae3-bb701565bd45 admin admin] PATH_INFO: `/v3/projects/6163225642c64821abc1d2e30f81400f` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:33.266683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926e55b9-edd6-4633-8ae3-bb701565bd45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6163225642c64821abc1d2e30f81400f)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:33.270245 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926e55b9-edd6-4633-8ae3-bb701565bd45 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:33.291044 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:15:33.398668 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-926e55b9-edd6-4633-8ae3-bb701565bd45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6163225642c64821abc1d2e30f81400f', 'request_id': 'req-926e55b9-edd6-4633-8ae3-bb701565bd45'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:33.497124 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-926e55b9-edd6-4633-8ae3-bb701565bd45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6163225642c64821abc1d2e30f81400f', 'request_id': 'req-926e55b9-edd6-4633-8ae3-bb701565bd45'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:33.504263 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-926e55b9-edd6-4633-8ae3-bb701565bd45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6163225642c64821abc1d2e30f81400f', 'request_id': 'req-926e55b9-edd6-4633-8ae3-bb701565bd45'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:33.517004 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-926e55b9-edd6-4633-8ae3-bb701565bd45 admin admin] The token cache is being invalidate because project 6163225642c64821abc1d2e30f81400f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81650) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:15:33.517844 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-926e55b9-edd6-4633-8ae3-bb701565bd45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:33.520065 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-926e55b9-edd6-4633-8ae3-bb701565bd45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:33.521860 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1195/2407] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:15:33 2022] DELETE /identity/v3/projects/6163225642c64821abc1d2e30f81400f => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:15:33.528590 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4cc294-dbbd-41dc-a427-eab95e0d9b5d None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:33.529142 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4cc294-dbbd-41dc-a427-eab95e0d9b5d None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:33.529142 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4cc294-dbbd-41dc-a427-eab95e0d9b5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:33.561066 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-ef4cc294-dbbd-41dc-a427-eab95e0d9b5d None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:15:33.562392 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-ef4cc294-dbbd-41dc-a427-eab95e0d9b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:33.601417 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:15:33.632257 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1213/2408] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:15:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:15:33.640945 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e298da41-68a8-4195-a50e-b2ea36cc2ae4 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:33.641360 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e298da41-68a8-4195-a50e-b2ea36cc2ae4 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:33.641850 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e298da41-68a8-4195-a50e-b2ea36cc2ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:33.662886 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-e298da41-68a8-4195-a50e-b2ea36cc2ae4 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:15:33.664229 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-e298da41-68a8-4195-a50e-b2ea36cc2ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:33.672094 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1196/2409] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:15:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:15:33.676685 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4cc294-dbbd-41dc-a427-eab95e0d9b5d None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:33.686107 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4cc294-dbbd-41dc-a427-eab95e0d9b5d None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:33.690693 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832d3100-1bcd-4e28-be43-33b7d93e90a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:33.691972 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832d3100-1bcd-4e28-be43-33b7d93e90a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:33.697344 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832d3100-1bcd-4e28-be43-33b7d93e90a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:33.697344 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832d3100-1bcd-4e28-be43-33b7d93e90a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:33.697344 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832d3100-1bcd-4e28-be43-33b7d93e90a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:15:33.697344 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832d3100-1bcd-4e28-be43-33b7d93e90a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:33.697344 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832d3100-1bcd-4e28-be43-33b7d93e90a1 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:33.706825 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1214/2410] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:15:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 17:15:33.711993 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e298da41-68a8-4195-a50e-b2ea36cc2ae4 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:33.726069 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e298da41-68a8-4195-a50e-b2ea36cc2ae4 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:33.728015 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed4b75-30be-41b9-8235-78269df6fa59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:33.728623 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed4b75-30be-41b9-8235-78269df6fa59 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:33.728911 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed4b75-30be-41b9-8235-78269df6fa59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:33.729183 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed4b75-30be-41b9-8235-78269df6fa59 admin admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:33.729805 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ed4b75-30be-41b9-8235-78269df6fa59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:33.731776 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ed4b75-30be-41b9-8235-78269df6fa59 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:33.834423 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832d3100-1bcd-4e28-be43-33b7d93e90a1 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:33.836222 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-832d3100-1bcd-4e28-be43-33b7d93e90a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:33.843548 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1197/2411] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:15:33 2022] POST /identity/v3/projects => generated 379 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:15:33.845535 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed4b75-30be-41b9-8235-78269df6fa59 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:33.938930 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed4b75-30be-41b9-8235-78269df6fa59 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:33.948612 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061a3b71-31ab-4ec1-a3ec-6cea40783d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:33.948954 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061a3b71-31ab-4ec1-a3ec-6cea40783d82 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:33.949250 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061a3b71-31ab-4ec1-a3ec-6cea40783d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:33.949524 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061a3b71-31ab-4ec1-a3ec-6cea40783d82 admin admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:33.950168 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061a3b71-31ab-4ec1-a3ec-6cea40783d82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:33.952639 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061a3b71-31ab-4ec1-a3ec-6cea40783d82 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:33.964914 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832d3100-1bcd-4e28-be43-33b7d93e90a1 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:34.031438 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1198/2412] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:15:33 2022] POST /identity/v3/users => generated 290 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:15:34.034578 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061a3b71-31ab-4ec1-a3ec-6cea40783d82 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:34.049804 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa2ab2e-edec-4535-8139-805e2b2fa3fe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29653ba1a2a74232972ef7a9000518b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:34.051892 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa2ab2e-edec-4535-8139-805e2b2fa3fe service placement] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:34.052113 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa2ab2e-edec-4535-8139-805e2b2fa3fe service placement] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:34.052336 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa2ab2e-edec-4535-8139-805e2b2fa3fe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:34.053134 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061a3b71-31ab-4ec1-a3ec-6cea40783d82 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:34.054439 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-8fa2ab2e-edec-4535-8139-805e2b2fa3fe service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:34.058593 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c14dc50-976f-4b5f-b6b3-7e291f8029e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:34.059474 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c14dc50-976f-4b5f-b6b3-7e291f8029e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:34.059841 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c14dc50-976f-4b5f-b6b3-7e291f8029e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:34.060248 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c14dc50-976f-4b5f-b6b3-7e291f8029e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:34.061110 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c14dc50-976f-4b5f-b6b3-7e291f8029e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:34.062955 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c14dc50-976f-4b5f-b6b3-7e291f8029e5 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:34.073322 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa2ab2e-edec-4535-8139-805e2b2fa3fe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:34.075130 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa2ab2e-edec-4535-8139-805e2b2fa3fe service placement] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:34.080547 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1199/2413] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:15:34.081697 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c14dc50-976f-4b5f-b6b3-7e291f8029e5 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:34.089888 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1215/2414] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:15:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 17:15:34.093962 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c14dc50-976f-4b5f-b6b3-7e291f8029e5 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:34.099750 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8fc8fd-420d-4228-80b8-96c77c209f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:34.100464 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8fc8fd-420d-4228-80b8-96c77c209f83 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:34.100750 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8fc8fd-420d-4228-80b8-96c77c209f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:34.101031 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8fc8fd-420d-4228-80b8-96c77c209f83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:34.101729 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8fc8fd-420d-4228-80b8-96c77c209f83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:34.103356 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8fc8fd-420d-4228-80b8-96c77c209f83 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:34.118054 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1200/2415] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:15:34.119164 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa2ab2e-edec-4535-8139-805e2b2fa3fe service placement] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:34.130626 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa2ab2e-edec-4535-8139-805e2b2fa3fe service placement] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:34.135821 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639aaa47-4e11-4c37-8fe3-317e0c0d5322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:34.136852 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639aaa47-4e11-4c37-8fe3-317e0c0d5322 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:34.137140 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639aaa47-4e11-4c37-8fe3-317e0c0d5322 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:34.137928 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639aaa47-4e11-4c37-8fe3-317e0c0d5322 admin admin] PATH_INFO: `/v3/projects/e976b3d87de347818a3c62fa4806db98/users/04ab53c336934355a0000bda59fa7e7c/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:34.155543 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639aaa47-4e11-4c37-8fe3-317e0c0d5322 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e976b3d87de347818a3c62fa4806db98, user_id=04ab53c336934355a0000bda59fa7e7c, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:34.158451 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639aaa47-4e11-4c37-8fe3-317e0c0d5322 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:34.317339 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1216/2416] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:15:34 2022] PUT /identity/v3/projects/e976b3d87de347818a3c62fa4806db98/users/04ab53c336934355a0000bda59fa7e7c/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:15:34.322250 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b93150-8818-471c-8c59-c1b7296ef568 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:34.326606 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b93150-8818-471c-8c59-c1b7296ef568 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:34.326606 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b93150-8818-471c-8c59-c1b7296ef568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:34.353233 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-07b93150-8818-471c-8c59-c1b7296ef568 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:15:34.355319 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-07b93150-8818-471c-8c59-c1b7296ef568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:34.427474 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:15:34.459173 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1201/2417] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:15:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:15:34.470678 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639aaa47-4e11-4c37-8fe3-317e0c0d5322 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:34.473874 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-639aaa47-4e11-4c37-8fe3-317e0c0d5322 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:34.526242 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:15:34.555132 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639aaa47-4e11-4c37-8fe3-317e0c0d5322 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:34.558656 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713b8076-ab4e-4204-81f3-4efc0b2a5569 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:34.559298 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713b8076-ab4e-4204-81f3-4efc0b2a5569 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:34.559613 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713b8076-ab4e-4204-81f3-4efc0b2a5569 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:34.559883 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713b8076-ab4e-4204-81f3-4efc0b2a5569 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:34.571781 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713b8076-ab4e-4204-81f3-4efc0b2a5569 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:34.573318 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713b8076-ab4e-4204-81f3-4efc0b2a5569 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:34.587403 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1217/2418] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:15:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:15:40.786937 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2227b50-491a-48a5-8ac2-b1dc778c8ec2 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:40.787479 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2227b50-491a-48a5-8ac2-b1dc778c8ec2 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:40.787836 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2227b50-491a-48a5-8ac2-b1dc778c8ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:40.853801 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-d2227b50-491a-48a5-8ac2-b1dc778c8ec2 None None] MFA Rules not processed for user `04ab53c336934355a0000bda59fa7e7c`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:15:40.855482 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-d2227b50-491a-48a5-8ac2-b1dc778c8ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:40.956127 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1202/2419] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:15:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:15:40.960454 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713b8076-ab4e-4204-81f3-4efc0b2a5569 service neutron] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:40.978704 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713b8076-ab4e-4204-81f3-4efc0b2a5569 service neutron] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:40.989893 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40030c5c-e5eb-4346-a8f8-93d3a0251d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:40.990807 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40030c5c-e5eb-4346-a8f8-93d3a0251d22 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:40.991130 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40030c5c-e5eb-4346-a8f8-93d3a0251d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:40.991454 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40030c5c-e5eb-4346-a8f8-93d3a0251d22 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:40.992461 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40030c5c-e5eb-4346-a8f8-93d3a0251d22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:40.994376 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40030c5c-e5eb-4346-a8f8-93d3a0251d22 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:41.119926 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1218/2420] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:15:40 2022] POST /identity/v3/projects => generated 379 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:15:41.124175 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2227b50-491a-48a5-8ac2-b1dc778c8ec2 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:41.155980 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2227b50-491a-48a5-8ac2-b1dc778c8ec2 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:41.237085 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2c9bd9-99b8-430d-97ca-58deb44edd89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:41.238059 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c9bd9-99b8-430d-97ca-58deb44edd89 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:41.238931 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c9bd9-99b8-430d-97ca-58deb44edd89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:41.239514 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c9bd9-99b8-430d-97ca-58deb44edd89 admin admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:41.240336 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2c9bd9-99b8-430d-97ca-58deb44edd89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:41.242604 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2c9bd9-99b8-430d-97ca-58deb44edd89 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:41.380255 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1203/2421] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:15:41 2022] POST /identity/v3/users => generated 296 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:15:41.384053 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40030c5c-e5eb-4346-a8f8-93d3a0251d22 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:41.403152 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40030c5c-e5eb-4346-a8f8-93d3a0251d22 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:41.411645 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d23845d-47df-4ff5-a6e0-8f524f5df74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:41.416008 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d23845d-47df-4ff5-a6e0-8f524f5df74b admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:41.416008 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d23845d-47df-4ff5-a6e0-8f524f5df74b admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:41.416008 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d23845d-47df-4ff5-a6e0-8f524f5df74b admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:41.416008 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d23845d-47df-4ff5-a6e0-8f524f5df74b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:41.416493 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d23845d-47df-4ff5-a6e0-8f524f5df74b admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:41.434494 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2c9bd9-99b8-430d-97ca-58deb44edd89 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:41.456867 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1219/2422] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:15:41 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:15:41.467281 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2c9bd9-99b8-430d-97ca-58deb44edd89 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:41.483573 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6d2983-3cb6-4fd8-bf9e-3887533942f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:41.484544 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d2983-3cb6-4fd8-bf9e-3887533942f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:41.484828 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d2983-3cb6-4fd8-bf9e-3887533942f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:41.485120 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d2983-3cb6-4fd8-bf9e-3887533942f1 admin admin] PATH_INFO: `/v3/projects/62071b8ec9c24c8d97dd6e14e677fdca/users/6aec60a6c3954bb0ac7375f01a5f84ef/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:41.502932 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6d2983-3cb6-4fd8-bf9e-3887533942f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62071b8ec9c24c8d97dd6e14e677fdca, user_id=6aec60a6c3954bb0ac7375f01a5f84ef, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:41.505391 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6d2983-3cb6-4fd8-bf9e-3887533942f1 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:41.632518 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1204/2423] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:15:41 2022] PUT /identity/v3/projects/62071b8ec9c24c8d97dd6e14e677fdca/users/6aec60a6c3954bb0ac7375f01a5f84ef/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:15:41.635662 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d23845d-47df-4ff5-a6e0-8f524f5df74b admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:41.651410 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d23845d-47df-4ff5-a6e0-8f524f5df74b admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:41.752597 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d03a03-a286-463f-a5c0-8026dbff9f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:41.753440 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d03a03-a286-463f-a5c0-8026dbff9f4c admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:41.753783 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d03a03-a286-463f-a5c0-8026dbff9f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:41.754722 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d03a03-a286-463f-a5c0-8026dbff9f4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:41.755790 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d03a03-a286-463f-a5c0-8026dbff9f4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:41.757682 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d03a03-a286-463f-a5c0-8026dbff9f4c admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:41.775570 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1220/2424] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:15:41 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:15:41.780009 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6d2983-3cb6-4fd8-bf9e-3887533942f1 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:41.792694 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6d2983-3cb6-4fd8-bf9e-3887533942f1 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:41.806495 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332b118f-75e6-4d55-ba57-572d41b66df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:41.806495 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b118f-75e6-4d55-ba57-572d41b66df5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:41.806495 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b118f-75e6-4d55-ba57-572d41b66df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:41.806495 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b118f-75e6-4d55-ba57-572d41b66df5 admin admin] PATH_INFO: `/v3/projects/62071b8ec9c24c8d97dd6e14e677fdca/users/6aec60a6c3954bb0ac7375f01a5f84ef/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:41.807342 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332b118f-75e6-4d55-ba57-572d41b66df5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62071b8ec9c24c8d97dd6e14e677fdca, user_id=6aec60a6c3954bb0ac7375f01a5f84ef, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:41.810770 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332b118f-75e6-4d55-ba57-572d41b66df5 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:41.827133 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1205/2425] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:15:41 2022] PUT /identity/v3/projects/62071b8ec9c24c8d97dd6e14e677fdca/users/6aec60a6c3954bb0ac7375f01a5f84ef/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:15:43.648032 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d03a03-a286-463f-a5c0-8026dbff9f4c admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:43.650238 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-f1d03a03-a286-463f-a5c0-8026dbff9f4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:43.665843 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d03a03-a286-463f-a5c0-8026dbff9f4c admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:43.729956 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d482e534-f003-4ad8-950c-3c41ef40274a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:43.730633 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d482e534-f003-4ad8-950c-3c41ef40274a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:43.731031 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d482e534-f003-4ad8-950c-3c41ef40274a admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:43.731228 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d482e534-f003-4ad8-950c-3c41ef40274a admin admin] PATH_INFO: `/v3/users/be74b483a7ea45a9b002dbab58ea0dba` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:43.732365 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d482e534-f003-4ad8-950c-3c41ef40274a admin admin] RBAC: Authorizing `identity:delete_user(user_id=be74b483a7ea45a9b002dbab58ea0dba)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:43.734623 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d482e534-f003-4ad8-950c-3c41ef40274a admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:43.949174 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:15:43.958200 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-d482e534-f003-4ad8-950c-3c41ef40274a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be74b483a7ea45a9b002dbab58ea0dba'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:44.026944 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-d482e534-f003-4ad8-950c-3c41ef40274a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be74b483a7ea45a9b002dbab58ea0dba'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:44.027738 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-d482e534-f003-4ad8-950c-3c41ef40274a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be74b483a7ea45a9b002dbab58ea0dba'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:44.028639 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-d482e534-f003-4ad8-950c-3c41ef40274a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be74b483a7ea45a9b002dbab58ea0dba'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:44.043934 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-d482e534-f003-4ad8-950c-3c41ef40274a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be74b483a7ea45a9b002dbab58ea0dba'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:44.087423 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1221/2426] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:15:43 2022] DELETE /identity/v3/users/be74b483a7ea45a9b002dbab58ea0dba => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:15:44.380970 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332b118f-75e6-4d55-ba57-572d41b66df5 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:44.382915 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-332b118f-75e6-4d55-ba57-572d41b66df5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:44.397221 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332b118f-75e6-4d55-ba57-572d41b66df5 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:44.425290 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:15:44.472012 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23072b53-5efb-4427-97a9-c97b5427733e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:44.472724 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23072b53-5efb-4427-97a9-c97b5427733e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:44.473043 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23072b53-5efb-4427-97a9-c97b5427733e admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:44.473410 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23072b53-5efb-4427-97a9-c97b5427733e admin admin] PATH_INFO: `/v3/projects/8066561e145b47b5aaec87b7d0643272` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:44.474690 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23072b53-5efb-4427-97a9-c97b5427733e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8066561e145b47b5aaec87b7d0643272)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:44.476731 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23072b53-5efb-4427-97a9-c97b5427733e admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:44.674654 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-23072b53-5efb-4427-97a9-c97b5427733e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8066561e145b47b5aaec87b7d0643272', 'request_id': 'req-23072b53-5efb-4427-97a9-c97b5427733e'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:44.756255 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-23072b53-5efb-4427-97a9-c97b5427733e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8066561e145b47b5aaec87b7d0643272', 'request_id': 'req-23072b53-5efb-4427-97a9-c97b5427733e'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:44.765309 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-23072b53-5efb-4427-97a9-c97b5427733e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8066561e145b47b5aaec87b7d0643272', 'request_id': 'req-23072b53-5efb-4427-97a9-c97b5427733e'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:44.789663 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-23072b53-5efb-4427-97a9-c97b5427733e admin admin] The token cache is being invalidate because project 8066561e145b47b5aaec87b7d0643272 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81650) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:15:44.789953 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-23072b53-5efb-4427-97a9-c97b5427733e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:44.790822 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-23072b53-5efb-4427-97a9-c97b5427733e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:44.793894 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1206/2427] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:15:44 2022] DELETE /identity/v3/projects/8066561e145b47b5aaec87b7d0643272 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:15:44.798130 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d482e534-f003-4ad8-950c-3c41ef40274a admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:44.809513 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d482e534-f003-4ad8-950c-3c41ef40274a admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:44.893120 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5952d2e3-ae39-4da7-a1a7-8413687a49cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:44.896278 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5952d2e3-ae39-4da7-a1a7-8413687a49cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:44.896823 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5952d2e3-ae39-4da7-a1a7-8413687a49cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:44.897332 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5952d2e3-ae39-4da7-a1a7-8413687a49cd admin admin] PATH_INFO: `/v3/users/284490df3764411c9aa15edbf78b1541` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:44.902118 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5952d2e3-ae39-4da7-a1a7-8413687a49cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=284490df3764411c9aa15edbf78b1541)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:44.905409 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5952d2e3-ae39-4da7-a1a7-8413687a49cd admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:45.176446 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-5952d2e3-ae39-4da7-a1a7-8413687a49cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '284490df3764411c9aa15edbf78b1541'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:45.266758 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-5952d2e3-ae39-4da7-a1a7-8413687a49cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '284490df3764411c9aa15edbf78b1541'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:45.267838 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-5952d2e3-ae39-4da7-a1a7-8413687a49cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '284490df3764411c9aa15edbf78b1541'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:45.269759 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-5952d2e3-ae39-4da7-a1a7-8413687a49cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '284490df3764411c9aa15edbf78b1541'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:45.287990 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-5952d2e3-ae39-4da7-a1a7-8413687a49cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '284490df3764411c9aa15edbf78b1541'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:45.331504 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1222/2428] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:15:44 2022] DELETE /identity/v3/users/284490df3764411c9aa15edbf78b1541 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:15:45.618047 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23072b53-5efb-4427-97a9-c97b5427733e admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:45.620156 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-23072b53-5efb-4427-97a9-c97b5427733e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:45.636546 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23072b53-5efb-4427-97a9-c97b5427733e admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:45.714166 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954d1b37-c024-4b73-aceb-115e768589c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:45.715068 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d1b37-c024-4b73-aceb-115e768589c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:45.715454 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d1b37-c024-4b73-aceb-115e768589c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:45.715831 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d1b37-c024-4b73-aceb-115e768589c5 admin admin] PATH_INFO: `/v3/projects/5260f07fd435401da7acf7778026585d` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:45.717094 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954d1b37-c024-4b73-aceb-115e768589c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5260f07fd435401da7acf7778026585d)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:45.719537 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954d1b37-c024-4b73-aceb-115e768589c5 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:45.880368 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-954d1b37-c024-4b73-aceb-115e768589c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5260f07fd435401da7acf7778026585d', 'request_id': 'req-954d1b37-c024-4b73-aceb-115e768589c5'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:45.952818 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-954d1b37-c024-4b73-aceb-115e768589c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5260f07fd435401da7acf7778026585d', 'request_id': 'req-954d1b37-c024-4b73-aceb-115e768589c5'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:45.967411 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:15:45.967411 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-954d1b37-c024-4b73-aceb-115e768589c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5260f07fd435401da7acf7778026585d', 'request_id': 'req-954d1b37-c024-4b73-aceb-115e768589c5'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:45.983082 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-954d1b37-c024-4b73-aceb-115e768589c5 admin admin] The token cache is being invalidate because project 5260f07fd435401da7acf7778026585d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81650) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:15:45.983855 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-954d1b37-c024-4b73-aceb-115e768589c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:45.984856 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-954d1b37-c024-4b73-aceb-115e768589c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:15:45.988234 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1207/2429] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:15:45 2022] DELETE /identity/v3/projects/5260f07fd435401da7acf7778026585d => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:15:45.994290 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601095d2-c65d-4811-bf26-34a1df794c1f None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:45.995253 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601095d2-c65d-4811-bf26-34a1df794c1f None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:45.995754 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601095d2-c65d-4811-bf26-34a1df794c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:46.030751 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-601095d2-c65d-4811-bf26-34a1df794c1f None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:15:46.031054 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-601095d2-c65d-4811-bf26-34a1df794c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:46.097433 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:15:46.143062 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1223/2430] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:15:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:15:46.152532 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae59912-1deb-45a7-bdb5-a449175fdf19 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:46.152820 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae59912-1deb-45a7-bdb5-a449175fdf19 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:46.153099 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae59912-1deb-45a7-bdb5-a449175fdf19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:46.181025 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-6ae59912-1deb-45a7-bdb5-a449175fdf19 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:15:46.182251 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-6ae59912-1deb-45a7-bdb5-a449175fdf19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:46.198170 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1208/2431] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:15:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:15:46.199081 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601095d2-c65d-4811-bf26-34a1df794c1f None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:46.212306 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601095d2-c65d-4811-bf26-34a1df794c1f None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:46.234083 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d681f0e8-2f76-4ed7-bda8-8dda7692787e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:46.235241 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d681f0e8-2f76-4ed7-bda8-8dda7692787e admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:46.235677 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d681f0e8-2f76-4ed7-bda8-8dda7692787e admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:46.236134 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d681f0e8-2f76-4ed7-bda8-8dda7692787e admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:46.244831 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d681f0e8-2f76-4ed7-bda8-8dda7692787e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:15:46.244831 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d681f0e8-2f76-4ed7-bda8-8dda7692787e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:46.244831 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d681f0e8-2f76-4ed7-bda8-8dda7692787e admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:46.263250 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae59912-1deb-45a7-bdb5-a449175fdf19 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:46.269785 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1224/2432] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:15:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 17:15:46.274176 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae59912-1deb-45a7-bdb5-a449175fdf19 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:46.280085 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a70d4f5-ada3-4325-92fb-3efd2e14f4ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:46.280725 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a70d4f5-ada3-4325-92fb-3efd2e14f4ff admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:46.281007 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a70d4f5-ada3-4325-92fb-3efd2e14f4ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:46.281288 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a70d4f5-ada3-4325-92fb-3efd2e14f4ff admin admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:46.281972 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a70d4f5-ada3-4325-92fb-3efd2e14f4ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:46.284473 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a70d4f5-ada3-4325-92fb-3efd2e14f4ff admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:46.422730 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1209/2433] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:15:46 2022] POST /identity/v3/projects => generated 373 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:15:46.427606 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d681f0e8-2f76-4ed7-bda8-8dda7692787e admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:46.439054 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d681f0e8-2f76-4ed7-bda8-8dda7692787e admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:46.510537 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5613aa-438e-42af-a991-6bd8ce40aa22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:46.511034 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5613aa-438e-42af-a991-6bd8ce40aa22 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:46.511318 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5613aa-438e-42af-a991-6bd8ce40aa22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:46.514522 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5613aa-438e-42af-a991-6bd8ce40aa22 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:46.515247 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5613aa-438e-42af-a991-6bd8ce40aa22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:46.517347 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5613aa-438e-42af-a991-6bd8ce40aa22 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:46.626683 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1225/2434] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:15:46 2022] POST /identity/v3/users => generated 287 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 17:15:46.630961 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a70d4f5-ada3-4325-92fb-3efd2e14f4ff admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:46.642508 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a70d4f5-ada3-4325-92fb-3efd2e14f4ff admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:46.648224 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce47f27c-cc99-412b-b15b-4b8e8881fe4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:46.651610 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce47f27c-cc99-412b-b15b-4b8e8881fe4e admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:46.651610 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce47f27c-cc99-412b-b15b-4b8e8881fe4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:46.651610 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce47f27c-cc99-412b-b15b-4b8e8881fe4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:46.651610 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce47f27c-cc99-412b-b15b-4b8e8881fe4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:46.651923 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce47f27c-cc99-412b-b15b-4b8e8881fe4e admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:46.662422 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1210/2435] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:15:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:15:46.666619 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5613aa-438e-42af-a991-6bd8ce40aa22 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:46.679020 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5613aa-438e-42af-a991-6bd8ce40aa22 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:46.688841 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5a7652-de43-4578-a947-8cec2fda326a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:46.689722 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a7652-de43-4578-a947-8cec2fda326a admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:46.689722 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a7652-de43-4578-a947-8cec2fda326a admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:46.690006 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a7652-de43-4578-a947-8cec2fda326a admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:46.690706 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5a7652-de43-4578-a947-8cec2fda326a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:46.692385 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5a7652-de43-4578-a947-8cec2fda326a admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:46.709436 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1226/2436] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:15:46 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:15:46.713774 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce47f27c-cc99-412b-b15b-4b8e8881fe4e admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:46.732454 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce47f27c-cc99-412b-b15b-4b8e8881fe4e admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:46.741411 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8a8551-5f79-4748-a337-37bcac243d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:46.741411 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a8551-5f79-4748-a337-37bcac243d7d admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:46.741411 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a8551-5f79-4748-a337-37bcac243d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:46.741411 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a8551-5f79-4748-a337-37bcac243d7d admin admin] PATH_INFO: `/v3/projects/5598723d6c6a4c948c9626b67e5e7a8f/users/94ac624de4df45ef8f664ea9399f57b7/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:46.774921 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8a8551-5f79-4748-a337-37bcac243d7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5598723d6c6a4c948c9626b67e5e7a8f, user_id=94ac624de4df45ef8f664ea9399f57b7, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:46.777841 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8a8551-5f79-4748-a337-37bcac243d7d admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:46.882781 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1211/2437] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:15:46 2022] PUT /identity/v3/projects/5598723d6c6a4c948c9626b67e5e7a8f/users/94ac624de4df45ef8f664ea9399f57b7/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:15:46.895617 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc043ce4-4e84-4b47-9d5d-4937a0ce473e None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:46.895906 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc043ce4-4e84-4b47-9d5d-4937a0ce473e None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:46.896550 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc043ce4-4e84-4b47-9d5d-4937a0ce473e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:46.936888 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-dc043ce4-4e84-4b47-9d5d-4937a0ce473e None None] MFA Rules not processed for user `94ac624de4df45ef8f664ea9399f57b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:15:46.938681 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-dc043ce4-4e84-4b47-9d5d-4937a0ce473e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:47.026358 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1227/2438] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:15:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:15:47.038373 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8a8551-5f79-4748-a337-37bcac243d7d admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:47.040843 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-9d8a8551-5f79-4748-a337-37bcac243d7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:47.105963 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8a8551-5f79-4748-a337-37bcac243d7d admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:47.112674 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bd2db4-b5dd-4c2a-8cd1-422a27bb11de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:47.113309 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd2db4-b5dd-4c2a-8cd1-422a27bb11de service cinder] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:47.113596 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd2db4-b5dd-4c2a-8cd1-422a27bb11de service cinder] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:47.113877 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd2db4-b5dd-4c2a-8cd1-422a27bb11de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:47.124346 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bd2db4-b5dd-4c2a-8cd1-422a27bb11de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:47.126377 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bd2db4-b5dd-4c2a-8cd1-422a27bb11de service cinder] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:47.138949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1212/2439] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:15:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:15:48.332318 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cde5e3-4b5a-4c2c-9e20-48b50cff7ab8 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:48.332817 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cde5e3-4b5a-4c2c-9e20-48b50cff7ab8 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:48.332911 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cde5e3-4b5a-4c2c-9e20-48b50cff7ab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:48.363118 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-48cde5e3-4b5a-4c2c-9e20-48b50cff7ab8 None None] MFA Rules not processed for user `6aec60a6c3954bb0ac7375f01a5f84ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:15:48.364263 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-48cde5e3-4b5a-4c2c-9e20-48b50cff7ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:48.383207 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:15:48.434822 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1228/2440] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:15:48 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:15:48.449727 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bd2db4-b5dd-4c2a-8cd1-422a27bb11de service cinder] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:48.451420 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-98bd2db4-b5dd-4c2a-8cd1-422a27bb11de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:48.531645 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bd2db4-b5dd-4c2a-8cd1-422a27bb11de service cinder] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:48.539962 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90b485d-b2d4-45e6-8a2f-dbff29a32548 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55723c756cae4e7ab636054db48de02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:48.540599 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b485d-b2d4-45e6-8a2f-dbff29a32548 service nova] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:48.540887 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b485d-b2d4-45e6-8a2f-dbff29a32548 service nova] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:48.541171 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b485d-b2d4-45e6-8a2f-dbff29a32548 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:48.543545 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-b90b485d-b2d4-45e6-8a2f-dbff29a32548 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:48.566718 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:INSERT=2 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:15:48.610401 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90b485d-b2d4-45e6-8a2f-dbff29a32548 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:48.612041 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90b485d-b2d4-45e6-8a2f-dbff29a32548 service nova] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:48.621850 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1213/2441] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:15:48 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:15:57.345409 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe95d4-670f-411e-a0fb-7c2ca612ceb2 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:57.345960 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe95d4-670f-411e-a0fb-7c2ca612ceb2 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:57.345960 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe95d4-670f-411e-a0fb-7c2ca612ceb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:57.366064 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-dbbe95d4-670f-411e-a0fb-7c2ca612ceb2 None None] MFA Rules not processed for user `5de19a54ed104bcdb35ee4e5e793f1e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:15:57.367546 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-dbbe95d4-670f-411e-a0fb-7c2ca612ceb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:57.425556 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1229/2442] 10.222.0.82 () {58 vars in 1009 bytes} [Thu Aug 18 17:15:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:15:57.471662 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90b485d-b2d4-45e6-8a2f-dbff29a32548 service nova] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:15:57.480438 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-b90b485d-b2d4-45e6-8a2f-dbff29a32548 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:15:57.556783 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90b485d-b2d4-45e6-8a2f-dbff29a32548 service nova] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:15:57.561892 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cae9fa-019f-44fd-b667-aa902c24dec6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1381673f81584f4b8084823e3aa99e2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:15:57.562530 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cae9fa-019f-44fd-b667-aa902c24dec6 service swift] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:15:57.562812 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cae9fa-019f-44fd-b667-aa902c24dec6 service swift] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:15:57.563117 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cae9fa-019f-44fd-b667-aa902c24dec6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:15:57.571183 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cae9fa-019f-44fd-b667-aa902c24dec6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:15:57.572781 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cae9fa-019f-44fd-b667-aa902c24dec6 service swift] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:15:57.581486 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1214/2443] 10.222.0.82 () {60 vars in 1384 bytes} [Thu Aug 18 17:15:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 17:16:07.396289 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbe95d4-670f-411e-a0fb-7c2ca612ceb2 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:16:07.486443 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbe95d4-670f-411e-a0fb-7c2ca612ceb2 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:16:07.498176 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ec15e4-2438-4d15-94bc-16145a34eeab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:16:07.499848 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ec15e4-2438-4d15-94bc-16145a34eeab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:16:07.499848 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ec15e4-2438-4d15-94bc-16145a34eeab admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:16:07.500256 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ec15e4-2438-4d15-94bc-16145a34eeab admin admin] PATH_INFO: `/v3/users/94ac624de4df45ef8f664ea9399f57b7` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:16:07.502086 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ec15e4-2438-4d15-94bc-16145a34eeab admin admin] RBAC: Authorizing `identity:delete_user(user_id=94ac624de4df45ef8f664ea9399f57b7)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:16:07.505129 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ec15e4-2438-4d15-94bc-16145a34eeab admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:16:07.577998 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=49 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:16:07.697275 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-76ec15e4-2438-4d15-94bc-16145a34eeab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94ac624de4df45ef8f664ea9399f57b7'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:16:07.773252 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-76ec15e4-2438-4d15-94bc-16145a34eeab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94ac624de4df45ef8f664ea9399f57b7'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:16:07.774558 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-76ec15e4-2438-4d15-94bc-16145a34eeab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94ac624de4df45ef8f664ea9399f57b7'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:16:07.775294 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-76ec15e4-2438-4d15-94bc-16145a34eeab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94ac624de4df45ef8f664ea9399f57b7'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:16:07.789128 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-76ec15e4-2438-4d15-94bc-16145a34eeab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94ac624de4df45ef8f664ea9399f57b7'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:16:07.791438 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:16:07.813397 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1230/2444] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:16:07 2022] DELETE /identity/v3/users/94ac624de4df45ef8f664ea9399f57b7 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:16:07.818031 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7446822-f2e0-44ba-83a6-7f0496ec848c None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:16:07.818768 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7446822-f2e0-44ba-83a6-7f0496ec848c None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:16:07.819152 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7446822-f2e0-44ba-83a6-7f0496ec848c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:16:07.841167 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-e7446822-f2e0-44ba-83a6-7f0496ec848c None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:16:07.842837 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-e7446822-f2e0-44ba-83a6-7f0496ec848c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:16:07.920628 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1215/2445] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:16:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:16:07.929994 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ec15e4-2438-4d15-94bc-16145a34eeab admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:16:07.932015 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-76ec15e4-2438-4d15-94bc-16145a34eeab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:16:08.006110 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ec15e4-2438-4d15-94bc-16145a34eeab admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:16:08.010692 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd20f9ae-cf5e-49e1-8d35-f23f4c046e8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:16:08.011365 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd20f9ae-cf5e-49e1-8d35-f23f4c046e8d service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:16:08.011656 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd20f9ae-cf5e-49e1-8d35-f23f4c046e8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:16:08.011939 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd20f9ae-cf5e-49e1-8d35-f23f4c046e8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:16:08.021480 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd20f9ae-cf5e-49e1-8d35-f23f4c046e8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:16:08.023038 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd20f9ae-cf5e-49e1-8d35-f23f4c046e8d service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:16:08.036212 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1231/2446] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:16:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:16:08.507726 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7446822-f2e0-44ba-83a6-7f0496ec848c None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:16:08.515204 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-e7446822-f2e0-44ba-83a6-7f0496ec848c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:16:08.542602 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7446822-f2e0-44ba-83a6-7f0496ec848c None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:16:08.553646 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdf7067-8ea3-4e61-8d83-16a2ccbfc412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:16:08.554964 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdf7067-8ea3-4e61-8d83-16a2ccbfc412 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:16:08.555714 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdf7067-8ea3-4e61-8d83-16a2ccbfc412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:16:08.556222 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdf7067-8ea3-4e61-8d83-16a2ccbfc412 admin admin] PATH_INFO: `/v3/projects/5598723d6c6a4c948c9626b67e5e7a8f` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:16:08.560895 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdf7067-8ea3-4e61-8d83-16a2ccbfc412 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5598723d6c6a4c948c9626b67e5e7a8f)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:16:08.563042 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdf7067-8ea3-4e61-8d83-16a2ccbfc412 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:16:08.695889 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-1bdf7067-8ea3-4e61-8d83-16a2ccbfc412 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5598723d6c6a4c948c9626b67e5e7a8f', 'request_id': 'req-1bdf7067-8ea3-4e61-8d83-16a2ccbfc412'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:16:08.784976 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-1bdf7067-8ea3-4e61-8d83-16a2ccbfc412 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5598723d6c6a4c948c9626b67e5e7a8f', 'request_id': 'req-1bdf7067-8ea3-4e61-8d83-16a2ccbfc412'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:16:08.793171 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-1bdf7067-8ea3-4e61-8d83-16a2ccbfc412 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5598723d6c6a4c948c9626b67e5e7a8f', 'request_id': 'req-1bdf7067-8ea3-4e61-8d83-16a2ccbfc412'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:16:08.814757 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-1bdf7067-8ea3-4e61-8d83-16a2ccbfc412 admin admin] The token cache is being invalidate because project 5598723d6c6a4c948c9626b67e5e7a8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81650) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:16:08.815100 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-1bdf7067-8ea3-4e61-8d83-16a2ccbfc412 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:16:08.815771 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-1bdf7067-8ea3-4e61-8d83-16a2ccbfc412 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:16:08.817765 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1216/2447] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:16:08 2022] DELETE /identity/v3/projects/5598723d6c6a4c948c9626b67e5e7a8f => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:16:08.822937 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef79604a-f074-4c97-bc3a-5a42e59c9983 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:16:08.823787 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef79604a-f074-4c97-bc3a-5a42e59c9983 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:16:08.824066 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef79604a-f074-4c97-bc3a-5a42e59c9983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:16:08.873738 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-ef79604a-f074-4c97-bc3a-5a42e59c9983 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:16:08.877317 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-ef79604a-f074-4c97-bc3a-5a42e59c9983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:16:08.945053 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1232/2448] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:16:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:16:08.953493 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4a8963-85d4-4a6f-89f4-d099015cc969 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:16:08.954084 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4a8963-85d4-4a6f-89f4-d099015cc969 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:16:08.954525 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4a8963-85d4-4a6f-89f4-d099015cc969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:16:08.977244 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-ef4a8963-85d4-4a6f-89f4-d099015cc969 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:16:08.979221 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-ef4a8963-85d4-4a6f-89f4-d099015cc969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:16:08.987660 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1217/2449] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:16:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:16:08.991995 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef79604a-f074-4c97-bc3a-5a42e59c9983 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:16:09.003497 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef79604a-f074-4c97-bc3a-5a42e59c9983 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:16:09.014219 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a37338e-38c2-47f9-ab6e-0f1a8eb08d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:16:09.014965 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a37338e-38c2-47f9-ab6e-0f1a8eb08d17 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:16:09.015350 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a37338e-38c2-47f9-ab6e-0f1a8eb08d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:16:09.015698 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a37338e-38c2-47f9-ab6e-0f1a8eb08d17 admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:16:09.016431 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a37338e-38c2-47f9-ab6e-0f1a8eb08d17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:16:09.016820 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a37338e-38c2-47f9-ab6e-0f1a8eb08d17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:16:09.018576 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a37338e-38c2-47f9-ab6e-0f1a8eb08d17 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:16:09.035557 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1233/2450] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:16:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 17:16:09.036679 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4a8963-85d4-4a6f-89f4-d099015cc969 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:16:09.047085 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4a8963-85d4-4a6f-89f4-d099015cc969 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:16:09.051950 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b2ca9e-0603-4ae4-b42d-0e4c8b91cee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:16:09.052648 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b2ca9e-0603-4ae4-b42d-0e4c8b91cee1 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:16:09.052945 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b2ca9e-0603-4ae4-b42d-0e4c8b91cee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:16:09.053221 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b2ca9e-0603-4ae4-b42d-0e4c8b91cee1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:16:09.053882 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b2ca9e-0603-4ae4-b42d-0e4c8b91cee1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:16:09.056432 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b2ca9e-0603-4ae4-b42d-0e4c8b91cee1 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:16:09.171465 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1218/2451] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:16:09 2022] POST /identity/v3/projects => generated 369 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:16:09.175626 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a37338e-38c2-47f9-ab6e-0f1a8eb08d17 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:16:09.188221 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a37338e-38c2-47f9-ab6e-0f1a8eb08d17 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:16:09.264020 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0963964-2af2-4f28-8f85-e813b8c26111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:16:09.264835 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0963964-2af2-4f28-8f85-e813b8c26111 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:16:09.265124 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0963964-2af2-4f28-8f85-e813b8c26111 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:16:09.265401 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0963964-2af2-4f28-8f85-e813b8c26111 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:16:09.266224 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0963964-2af2-4f28-8f85-e813b8c26111 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:16:09.268962 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0963964-2af2-4f28-8f85-e813b8c26111 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:16:09.348678 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:16:09.355644 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1234/2452] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:16:09 2022] POST /identity/v3/users => generated 285 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:16:09.359593 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b2ca9e-0603-4ae4-b42d-0e4c8b91cee1 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:16:09.373737 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b2ca9e-0603-4ae4-b42d-0e4c8b91cee1 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:16:09.380648 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6b9f1f-1693-4acd-ab69-fa981f2e4f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:16:09.381607 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6b9f1f-1693-4acd-ab69-fa981f2e4f54 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:16:09.381985 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6b9f1f-1693-4acd-ab69-fa981f2e4f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:16:09.382348 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6b9f1f-1693-4acd-ab69-fa981f2e4f54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:16:09.383323 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6b9f1f-1693-4acd-ab69-fa981f2e4f54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:16:09.385617 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6b9f1f-1693-4acd-ab69-fa981f2e4f54 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:16:09.401820 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1219/2453] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:16:09 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 17:16:09.403049 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0963964-2af2-4f28-8f85-e813b8c26111 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:16:09.414153 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0963964-2af2-4f28-8f85-e813b8c26111 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:16:09.420896 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e12d82f-6703-40f0-bad3-6589f3b2aa77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:16:09.421537 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e12d82f-6703-40f0-bad3-6589f3b2aa77 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:16:09.421824 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e12d82f-6703-40f0-bad3-6589f3b2aa77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:16:09.422101 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e12d82f-6703-40f0-bad3-6589f3b2aa77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:16:09.422755 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e12d82f-6703-40f0-bad3-6589f3b2aa77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:16:09.424508 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e12d82f-6703-40f0-bad3-6589f3b2aa77 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:16:09.443568 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1235/2454] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:16:09 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:16:09.448915 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6b9f1f-1693-4acd-ab69-fa981f2e4f54 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:16:09.460954 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6b9f1f-1693-4acd-ab69-fa981f2e4f54 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:16:09.465793 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7f313d-0981-4ac0-951a-5d85a56bc387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:16:09.466449 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f313d-0981-4ac0-951a-5d85a56bc387 admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:16:09.466732 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f313d-0981-4ac0-951a-5d85a56bc387 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:16:09.467273 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f313d-0981-4ac0-951a-5d85a56bc387 admin admin] PATH_INFO: `/v3/projects/bc55c90151a44afab538eb52a8bd9204/users/e98b8a658a7d49768032534bf057654e/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:16:09.482580 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7f313d-0981-4ac0-951a-5d85a56bc387 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc55c90151a44afab538eb52a8bd9204, user_id=e98b8a658a7d49768032534bf057654e, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:16:09.485322 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7f313d-0981-4ac0-951a-5d85a56bc387 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:16:09.606778 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1220/2455] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:16:09 2022] PUT /identity/v3/projects/bc55c90151a44afab538eb52a8bd9204/users/e98b8a658a7d49768032534bf057654e/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:16:09.621962 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0459eb98-8ba9-4b01-afde-9ff984c57804 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:16:09.622265 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0459eb98-8ba9-4b01-afde-9ff984c57804 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:16:09.622541 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0459eb98-8ba9-4b01-afde-9ff984c57804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:16:09.658941 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-0459eb98-8ba9-4b01-afde-9ff984c57804 None None] MFA Rules not processed for user `e98b8a658a7d49768032534bf057654e`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:16:09.658941 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-0459eb98-8ba9-4b01-afde-9ff984c57804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:16:09.770594 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1236/2456] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:16:09 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:16:17.934962 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7f313d-0981-4ac0-951a-5d85a56bc387 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:16:17.936644 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-bf7f313d-0981-4ac0-951a-5d85a56bc387 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:16:17.962915 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:16:18.005592 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7f313d-0981-4ac0-951a-5d85a56bc387 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:16:18.011267 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5699088-cb59-424c-8258-e253357d0e17 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bee1ba6bb324de4915904f83349b3c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:16:18.011975 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5699088-cb59-424c-8258-e253357d0e17 service glance] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:16:18.012427 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5699088-cb59-424c-8258-e253357d0e17 service glance] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:16:18.012524 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5699088-cb59-424c-8258-e253357d0e17 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:16:18.021652 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5699088-cb59-424c-8258-e253357d0e17 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:16:18.023156 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5699088-cb59-424c-8258-e253357d0e17 service glance] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:16:18.033404 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1221/2457] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:16:17 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:16:19.100264 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93296a69-ddfc-4e70-b927-dbedd26a649b None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:16:19.100857 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93296a69-ddfc-4e70-b927-dbedd26a649b None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:16:19.100857 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93296a69-ddfc-4e70-b927-dbedd26a649b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:16:19.121471 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-93296a69-ddfc-4e70-b927-dbedd26a649b None None] MFA Rules not processed for user `5de19a54ed104bcdb35ee4e5e793f1e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:16:19.122633 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-93296a69-ddfc-4e70-b927-dbedd26a649b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:16:19.179941 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1237/2458] 10.222.0.82 () {58 vars in 967 bytes} [Thu Aug 18 17:16:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:16:19.193619 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5699088-cb59-424c-8258-e253357d0e17 service glance] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:16:19.200323 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-c5699088-cb59-424c-8258-e253357d0e17 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:16:19.268813 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5699088-cb59-424c-8258-e253357d0e17 service glance] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:16:19.274595 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed80bdd0-f5ec-461d-9ea2-91d888345aa5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1381673f81584f4b8084823e3aa99e2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:16:19.275362 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed80bdd0-f5ec-461d-9ea2-91d888345aa5 service swift] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:16:19.275644 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed80bdd0-f5ec-461d-9ea2-91d888345aa5 service swift] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:16:19.275916 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed80bdd0-f5ec-461d-9ea2-91d888345aa5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:16:19.284306 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed80bdd0-f5ec-461d-9ea2-91d888345aa5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:16:19.285920 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed80bdd0-f5ec-461d-9ea2-91d888345aa5 service swift] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:16:19.295284 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1222/2459] 10.222.0.82 () {60 vars in 1384 bytes} [Thu Aug 18 17:16:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 17:16:22.974495 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93296a69-ddfc-4e70-b927-dbedd26a649b None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:16:22.977343 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-93296a69-ddfc-4e70-b927-dbedd26a649b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:16:23.056486 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93296a69-ddfc-4e70-b927-dbedd26a649b None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:16:23.061731 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb5429-a584-49e0-a2d2-308da245196c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:16:23.071208 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb5429-a584-49e0-a2d2-308da245196c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:16:23.071208 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb5429-a584-49e0-a2d2-308da245196c admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:16:23.071208 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb5429-a584-49e0-a2d2-308da245196c admin admin] PATH_INFO: `/v3/users/497f5cef83ff4fe69e00e3ccf703088d` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:16:23.071208 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beb5429-a584-49e0-a2d2-308da245196c admin admin] RBAC: Authorizing `identity:delete_user(user_id=497f5cef83ff4fe69e00e3ccf703088d)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:16:23.071208 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beb5429-a584-49e0-a2d2-308da245196c admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:16:23.171889 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:16:23.284973 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-0beb5429-a584-49e0-a2d2-308da245196c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '497f5cef83ff4fe69e00e3ccf703088d'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:16:23.376455 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-0beb5429-a584-49e0-a2d2-308da245196c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '497f5cef83ff4fe69e00e3ccf703088d'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:16:23.377616 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-0beb5429-a584-49e0-a2d2-308da245196c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '497f5cef83ff4fe69e00e3ccf703088d'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:16:23.378769 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-0beb5429-a584-49e0-a2d2-308da245196c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '497f5cef83ff4fe69e00e3ccf703088d'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:16:23.395854 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-0beb5429-a584-49e0-a2d2-308da245196c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '497f5cef83ff4fe69e00e3ccf703088d'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:16:23.418514 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1238/2460] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:16:22 2022] DELETE /identity/v3/users/497f5cef83ff4fe69e00e3ccf703088d => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:16:23.748988 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed80bdd0-f5ec-461d-9ea2-91d888345aa5 service swift] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:16:23.776034 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed80bdd0-f5ec-461d-9ea2-91d888345aa5 service swift] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:16:23.848576 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4446f3-31ca-4a5f-bd28-ed8c72dd48a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:16:23.849238 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4446f3-31ca-4a5f-bd28-ed8c72dd48a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:16:23.849527 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4446f3-31ca-4a5f-bd28-ed8c72dd48a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:16:23.849823 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4446f3-31ca-4a5f-bd28-ed8c72dd48a6 admin admin] PATH_INFO: `/v3/projects/9e161860d18f4d608a9ce4177cce7b6f` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:16:23.851045 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4446f3-31ca-4a5f-bd28-ed8c72dd48a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e161860d18f4d608a9ce4177cce7b6f)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:16:23.854968 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4446f3-31ca-4a5f-bd28-ed8c72dd48a6 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:16:23.952081 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-2a4446f3-31ca-4a5f-bd28-ed8c72dd48a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e161860d18f4d608a9ce4177cce7b6f', 'request_id': 'req-2a4446f3-31ca-4a5f-bd28-ed8c72dd48a6'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:16:23.956639 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:16:24.013270 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-2a4446f3-31ca-4a5f-bd28-ed8c72dd48a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e161860d18f4d608a9ce4177cce7b6f', 'request_id': 'req-2a4446f3-31ca-4a5f-bd28-ed8c72dd48a6'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:16:24.019454 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-2a4446f3-31ca-4a5f-bd28-ed8c72dd48a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e161860d18f4d608a9ce4177cce7b6f', 'request_id': 'req-2a4446f3-31ca-4a5f-bd28-ed8c72dd48a6'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:16:24.032730 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-2a4446f3-31ca-4a5f-bd28-ed8c72dd48a6 admin admin] The token cache is being invalidate because project 9e161860d18f4d608a9ce4177cce7b6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81650) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:16:24.033072 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-2a4446f3-31ca-4a5f-bd28-ed8c72dd48a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:16:24.033728 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-2a4446f3-31ca-4a5f-bd28-ed8c72dd48a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:16:24.035364 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1223/2461] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:16:23 2022] DELETE /identity/v3/projects/9e161860d18f4d608a9ce4177cce7b6f => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:16:33.411844 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:DELETE=6,keystone:SELECT=19,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:16:34.104525 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=6,keystone:DELETE=2 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:16:44.139497 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8828ce-85aa-4568-932b-00f6a1714df0 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:16:44.140054 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8828ce-85aa-4568-932b-00f6a1714df0 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:16:44.140054 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8828ce-85aa-4568-932b-00f6a1714df0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:16:44.159152 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-1b8828ce-85aa-4568-932b-00f6a1714df0 None None] MFA Rules not processed for user `04ab53c336934355a0000bda59fa7e7c`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:16:44.160368 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-1b8828ce-85aa-4568-932b-00f6a1714df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:16:44.219510 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1239/2462] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:16:44 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:16:50.496902 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d954fc-e408-4de7-897d-eeb26e1102a4 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:16:50.497394 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d954fc-e408-4de7-897d-eeb26e1102a4 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:16:50.497394 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d954fc-e408-4de7-897d-eeb26e1102a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:16:50.518970 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-92d954fc-e408-4de7-897d-eeb26e1102a4 None None] MFA Rules not processed for user `32f58f87972d46be9496e6216f1f5ca1`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:16:50.522653 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-92d954fc-e408-4de7-897d-eeb26e1102a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:16:50.589189 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1224/2463] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:16:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 17:16:54.212508 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=25 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:16:54.781332 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8828ce-85aa-4568-932b-00f6a1714df0 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:16:54.782998 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-1b8828ce-85aa-4568-932b-00f6a1714df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:16:54.880064 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8828ce-85aa-4568-932b-00f6a1714df0 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:16:54.884380 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a454d64-1fe4-45a6-b836-969b6713415f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:16:54.884991 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a454d64-1fe4-45a6-b836-969b6713415f service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:16:54.885276 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a454d64-1fe4-45a6-b836-969b6713415f service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:16:54.885550 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a454d64-1fe4-45a6-b836-969b6713415f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:16:54.887328 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-7a454d64-1fe4-45a6-b836-969b6713415f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:16:54.975155 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a454d64-1fe4-45a6-b836-969b6713415f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:16:54.978648 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a454d64-1fe4-45a6-b836-969b6713415f service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:16:54.989236 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1240/2464] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:16:54 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:17:00.582494 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=25 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:17:04.985112 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=50 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:17:13.952500 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d954fc-e408-4de7-897d-eeb26e1102a4 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:17:13.953918 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-92d954fc-e408-4de7-897d-eeb26e1102a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:17:14.019044 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d954fc-e408-4de7-897d-eeb26e1102a4 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:17:14.023653 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99103ac1-d73b-4567-9598-7d8b4195ffdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:17:14.024291 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99103ac1-d73b-4567-9598-7d8b4195ffdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:17:14.024570 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99103ac1-d73b-4567-9598-7d8b4195ffdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:17:14.024842 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99103ac1-d73b-4567-9598-7d8b4195ffdd admin admin] PATH_INFO: `/v3/users/e98b8a658a7d49768032534bf057654e` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:17:14.025980 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99103ac1-d73b-4567-9598-7d8b4195ffdd admin admin] RBAC: Authorizing `identity:delete_user(user_id=e98b8a658a7d49768032534bf057654e)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:17:14.027950 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99103ac1-d73b-4567-9598-7d8b4195ffdd admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:17:14.249013 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-99103ac1-d73b-4567-9598-7d8b4195ffdd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e98b8a658a7d49768032534bf057654e'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:17:14.263453 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-99103ac1-d73b-4567-9598-7d8b4195ffdd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e98b8a658a7d49768032534bf057654e'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:17:14.264416 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-99103ac1-d73b-4567-9598-7d8b4195ffdd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e98b8a658a7d49768032534bf057654e'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:17:14.354154 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-99103ac1-d73b-4567-9598-7d8b4195ffdd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e98b8a658a7d49768032534bf057654e'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:17:14.354803 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-99103ac1-d73b-4567-9598-7d8b4195ffdd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e98b8a658a7d49768032534bf057654e'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:17:14.369073 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1225/2465] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:17:13 2022] DELETE /identity/v3/users/e98b8a658a7d49768032534bf057654e => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:17:14.374095 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fda428-4294-4351-b168-c2d04af24528 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:17:14.374616 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fda428-4294-4351-b168-c2d04af24528 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:17:14.374812 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fda428-4294-4351-b168-c2d04af24528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:17:14.396938 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-81fda428-4294-4351-b168-c2d04af24528 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:17:14.401517 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-81fda428-4294-4351-b168-c2d04af24528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:17:14.461274 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1241/2466] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:17:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:17:14.471325 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99103ac1-d73b-4567-9598-7d8b4195ffdd admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:17:14.472867 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-99103ac1-d73b-4567-9598-7d8b4195ffdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:17:14.533110 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99103ac1-d73b-4567-9598-7d8b4195ffdd admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:17:14.542317 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30af1b1-c667-4b07-a046-e0419d744614 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:17:14.542995 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30af1b1-c667-4b07-a046-e0419d744614 service neutron] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:17:14.543286 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30af1b1-c667-4b07-a046-e0419d744614 service neutron] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:17:14.543566 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30af1b1-c667-4b07-a046-e0419d744614 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:17:14.554002 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30af1b1-c667-4b07-a046-e0419d744614 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:17:14.555506 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30af1b1-c667-4b07-a046-e0419d744614 service neutron] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:17:14.565924 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1226/2467] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:17:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:17:14.953440 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fda428-4294-4351-b168-c2d04af24528 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:17:14.954878 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-81fda428-4294-4351-b168-c2d04af24528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:17:14.968403 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fda428-4294-4351-b168-c2d04af24528 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:17:14.972277 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b316cc0-4b46-4fda-ad03-fe1bc58f6ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:17:14.972914 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b316cc0-4b46-4fda-ad03-fe1bc58f6ac9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:17:14.973190 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b316cc0-4b46-4fda-ad03-fe1bc58f6ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:17:14.973468 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b316cc0-4b46-4fda-ad03-fe1bc58f6ac9 admin admin] PATH_INFO: `/v3/projects/bc55c90151a44afab538eb52a8bd9204` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:17:14.974567 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b316cc0-4b46-4fda-ad03-fe1bc58f6ac9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc55c90151a44afab538eb52a8bd9204)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:17:14.977326 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b316cc0-4b46-4fda-ad03-fe1bc58f6ac9 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:17:15.088395 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-2b316cc0-4b46-4fda-ad03-fe1bc58f6ac9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc55c90151a44afab538eb52a8bd9204', 'request_id': 'req-2b316cc0-4b46-4fda-ad03-fe1bc58f6ac9'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:17:15.095506 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-2b316cc0-4b46-4fda-ad03-fe1bc58f6ac9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc55c90151a44afab538eb52a8bd9204', 'request_id': 'req-2b316cc0-4b46-4fda-ad03-fe1bc58f6ac9'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:17:15.163065 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-2b316cc0-4b46-4fda-ad03-fe1bc58f6ac9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc55c90151a44afab538eb52a8bd9204', 'request_id': 'req-2b316cc0-4b46-4fda-ad03-fe1bc58f6ac9'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:17:15.175125 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-2b316cc0-4b46-4fda-ad03-fe1bc58f6ac9 admin admin] The token cache is being invalidate because project bc55c90151a44afab538eb52a8bd9204 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:17:15.175422 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-2b316cc0-4b46-4fda-ad03-fe1bc58f6ac9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:17:15.176068 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-2b316cc0-4b46-4fda-ad03-fe1bc58f6ac9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:17:15.177750 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1242/2468] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:17:14 2022] DELETE /identity/v3/projects/bc55c90151a44afab538eb52a8bd9204 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:17:15.182696 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5198b5-3b1d-4b6c-848f-fbc6b3f92951 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:17:15.183216 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5198b5-3b1d-4b6c-848f-fbc6b3f92951 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:17:15.183313 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5198b5-3b1d-4b6c-848f-fbc6b3f92951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:17:15.204166 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-2b5198b5-3b1d-4b6c-848f-fbc6b3f92951 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:17:15.205360 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-2b5198b5-3b1d-4b6c-848f-fbc6b3f92951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:17:15.223487 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:17:15.266240 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1227/2469] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:17:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:17:15.274202 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12b928b-d3ff-4aa6-9408-c8378c7940fd None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:17:15.274611 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12b928b-d3ff-4aa6-9408-c8378c7940fd None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:17:15.275028 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12b928b-d3ff-4aa6-9408-c8378c7940fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:17:15.295989 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-d12b928b-d3ff-4aa6-9408-c8378c7940fd None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:17:15.297359 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-d12b928b-d3ff-4aa6-9408-c8378c7940fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:17:15.305742 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1243/2470] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:17:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:17:15.310272 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5198b5-3b1d-4b6c-848f-fbc6b3f92951 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:17:15.320255 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5198b5-3b1d-4b6c-848f-fbc6b3f92951 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:17:15.325158 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86be5793-cbfa-48a3-9ca5-a4b8f0365212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:17:15.326325 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be5793-cbfa-48a3-9ca5-a4b8f0365212 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:17:15.326806 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be5793-cbfa-48a3-9ca5-a4b8f0365212 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:17:15.327343 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be5793-cbfa-48a3-9ca5-a4b8f0365212 admin admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:17:15.333077 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86be5793-cbfa-48a3-9ca5-a4b8f0365212 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:17:15.333077 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86be5793-cbfa-48a3-9ca5-a4b8f0365212 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:17:15.333077 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86be5793-cbfa-48a3-9ca5-a4b8f0365212 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:17:15.342303 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1228/2471] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:17:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 17:17:15.346749 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12b928b-d3ff-4aa6-9408-c8378c7940fd None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:17:15.356305 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12b928b-d3ff-4aa6-9408-c8378c7940fd None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:17:15.361296 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dfa7bb-6e93-42a4-be6a-a8e490b14213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:17:15.361920 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dfa7bb-6e93-42a4-be6a-a8e490b14213 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:17:15.362204 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dfa7bb-6e93-42a4-be6a-a8e490b14213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:17:15.362478 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dfa7bb-6e93-42a4-be6a-a8e490b14213 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:17:15.363132 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dfa7bb-6e93-42a4-be6a-a8e490b14213 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:17:15.365092 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dfa7bb-6e93-42a4-be6a-a8e490b14213 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:17:15.485065 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1244/2472] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:17:15 2022] POST /identity/v3/projects => generated 365 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:17:15.489461 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86be5793-cbfa-48a3-9ca5-a4b8f0365212 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:17:15.500250 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86be5793-cbfa-48a3-9ca5-a4b8f0365212 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:17:15.556868 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2755b6bc-8bc7-4ac6-860e-daa9adcde578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:17:15.557484 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2755b6bc-8bc7-4ac6-860e-daa9adcde578 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:17:15.557769 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2755b6bc-8bc7-4ac6-860e-daa9adcde578 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:17:15.558043 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2755b6bc-8bc7-4ac6-860e-daa9adcde578 admin admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:17:15.558667 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2755b6bc-8bc7-4ac6-860e-daa9adcde578 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:17:15.560845 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2755b6bc-8bc7-4ac6-860e-daa9adcde578 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:17:15.665045 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1229/2473] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:17:15 2022] POST /identity/v3/users => generated 283 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:17:15.669199 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dfa7bb-6e93-42a4-be6a-a8e490b14213 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:17:15.680775 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dfa7bb-6e93-42a4-be6a-a8e490b14213 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:17:15.686148 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac5288b-d6af-49c2-9803-f4226cb9d4f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:17:15.686754 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac5288b-d6af-49c2-9803-f4226cb9d4f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:17:15.687074 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac5288b-d6af-49c2-9803-f4226cb9d4f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:17:15.687347 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac5288b-d6af-49c2-9803-f4226cb9d4f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:17:15.687987 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac5288b-d6af-49c2-9803-f4226cb9d4f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:17:15.691160 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac5288b-d6af-49c2-9803-f4226cb9d4f6 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:17:15.703147 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1245/2474] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:17:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:17:15.706915 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2755b6bc-8bc7-4ac6-860e-daa9adcde578 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:17:15.719757 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2755b6bc-8bc7-4ac6-860e-daa9adcde578 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:17:15.724961 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f94fb7-36ed-4990-9f89-184c85289840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:17:15.725658 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f94fb7-36ed-4990-9f89-184c85289840 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:17:15.726051 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f94fb7-36ed-4990-9f89-184c85289840 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:17:15.726406 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f94fb7-36ed-4990-9f89-184c85289840 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:17:15.727202 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f94fb7-36ed-4990-9f89-184c85289840 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:17:15.728932 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f94fb7-36ed-4990-9f89-184c85289840 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:17:15.743432 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1230/2475] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:17:15 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:17:15.748560 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac5288b-d6af-49c2-9803-f4226cb9d4f6 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:17:15.758855 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac5288b-d6af-49c2-9803-f4226cb9d4f6 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:17:15.768642 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d687ccd8-0494-4f5d-8ca1-4e4cc31aa595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:17:15.769372 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687ccd8-0494-4f5d-8ca1-4e4cc31aa595 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:17:15.769661 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687ccd8-0494-4f5d-8ca1-4e4cc31aa595 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:17:15.770044 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687ccd8-0494-4f5d-8ca1-4e4cc31aa595 admin admin] PATH_INFO: `/v3/projects/da53e08509254c14b51831321cc7dc5f/users/bf9c01c6d32a493884eb279115c9078a/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:17:15.788462 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d687ccd8-0494-4f5d-8ca1-4e4cc31aa595 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da53e08509254c14b51831321cc7dc5f, user_id=bf9c01c6d32a493884eb279115c9078a, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:17:15.791853 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d687ccd8-0494-4f5d-8ca1-4e4cc31aa595 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:17:15.893140 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1246/2476] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:17:15 2022] PUT /identity/v3/projects/da53e08509254c14b51831321cc7dc5f/users/bf9c01c6d32a493884eb279115c9078a/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:17:15.902441 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988cda13-273b-44b6-973b-3b5b2d534d37 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:17:15.902807 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988cda13-273b-44b6-973b-3b5b2d534d37 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:17:15.903199 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988cda13-273b-44b6-973b-3b5b2d534d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:17:15.934132 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-988cda13-273b-44b6-973b-3b5b2d534d37 None None] MFA Rules not processed for user `bf9c01c6d32a493884eb279115c9078a`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:17:15.936214 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-988cda13-273b-44b6-973b-3b5b2d534d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:17:16.002543 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1231/2477] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:17:15 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:17:16.013965 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d687ccd8-0494-4f5d-8ca1-4e4cc31aa595 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:17:16.015755 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-d687ccd8-0494-4f5d-8ca1-4e4cc31aa595 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:17:16.037391 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:17:16.074092 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d687ccd8-0494-4f5d-8ca1-4e4cc31aa595 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:17:16.078095 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646cd80a-5ddc-459f-b996-136d6da062fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:17:16.078769 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646cd80a-5ddc-459f-b996-136d6da062fa service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:17:16.079082 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646cd80a-5ddc-459f-b996-136d6da062fa service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:17:16.079363 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646cd80a-5ddc-459f-b996-136d6da062fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:17:16.087653 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646cd80a-5ddc-459f-b996-136d6da062fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:17:16.089100 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646cd80a-5ddc-459f-b996-136d6da062fa service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:17:16.098061 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1247/2478] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:17:16 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:17:20.488021 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988cda13-273b-44b6-973b-3b5b2d534d37 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:17:20.489502 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-988cda13-273b-44b6-973b-3b5b2d534d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:17:20.509402 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:17:20.546374 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988cda13-273b-44b6-973b-3b5b2d534d37 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:17:20.550044 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f13c1f-220c-44e5-be25-1cd2be44c75d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:17:20.550966 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f13c1f-220c-44e5-be25-1cd2be44c75d service neutron] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:17:20.551247 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f13c1f-220c-44e5-be25-1cd2be44c75d service neutron] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:17:20.551519 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f13c1f-220c-44e5-be25-1cd2be44c75d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:17:20.553285 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-f1f13c1f-220c-44e5-be25-1cd2be44c75d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:17:20.622851 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f13c1f-220c-44e5-be25-1cd2be44c75d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:17:20.625123 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f13c1f-220c-44e5-be25-1cd2be44c75d service neutron] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:17:20.635782 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1232/2479] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:17:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:17:21.105546 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646cd80a-5ddc-459f-b996-136d6da062fa service cinder] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:17:21.115837 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646cd80a-5ddc-459f-b996-136d6da062fa service cinder] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:17:21.119403 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef13b1a-2b13-4f5a-81c4-fc7407690b96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:17:21.119990 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef13b1a-2b13-4f5a-81c4-fc7407690b96 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:17:21.120267 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef13b1a-2b13-4f5a-81c4-fc7407690b96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:17:21.120538 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef13b1a-2b13-4f5a-81c4-fc7407690b96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:17:21.122274 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-eef13b1a-2b13-4f5a-81c4-fc7407690b96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:17:21.135556 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef13b1a-2b13-4f5a-81c4-fc7407690b96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:17:21.136953 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef13b1a-2b13-4f5a-81c4-fc7407690b96 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:17:21.148235 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1248/2480] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:17:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:17:23.104355 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15884182-1e89-48fb-8128-35daa0c02be9 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:17:23.108091 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15884182-1e89-48fb-8128-35daa0c02be9 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:17:23.108091 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15884182-1e89-48fb-8128-35daa0c02be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:17:23.126104 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-15884182-1e89-48fb-8128-35daa0c02be9 None None] MFA Rules not processed for user `5de19a54ed104bcdb35ee4e5e793f1e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:17:23.127591 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-15884182-1e89-48fb-8128-35daa0c02be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:17:23.191621 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1233/2481] 10.222.0.82 () {58 vars in 1009 bytes} [Thu Aug 18 17:17:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:17:23.291935 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef13b1a-2b13-4f5a-81c4-fc7407690b96 service neutron] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:17:23.303974 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef13b1a-2b13-4f5a-81c4-fc7407690b96 service neutron] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:17:23.306263 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15884182-1e89-48fb-8128-35daa0c02be9 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:17:23.307801 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-15884182-1e89-48fb-8128-35daa0c02be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:17:23.308180 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96885317-cde8-4573-9e2b-2ab06e5ac241 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:17:23.308859 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96885317-cde8-4573-9e2b-2ab06e5ac241 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:17:23.309147 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96885317-cde8-4573-9e2b-2ab06e5ac241 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:17:23.309423 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96885317-cde8-4573-9e2b-2ab06e5ac241 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:17:23.311466 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-96885317-cde8-4573-9e2b-2ab06e5ac241 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:17:23.384426 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15884182-1e89-48fb-8128-35daa0c02be9 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:17:23.387188 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96885317-cde8-4573-9e2b-2ab06e5ac241 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:17:23.388635 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96885317-cde8-4573-9e2b-2ab06e5ac241 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:17:23.390639 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5d8a93-c3e4-4d3a-8926-56103918d30d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1381673f81584f4b8084823e3aa99e2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:17:23.391266 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d8a93-c3e4-4d3a-8926-56103918d30d service swift] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:17:23.391548 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d8a93-c3e4-4d3a-8926-56103918d30d service swift] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:17:23.391825 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d8a93-c3e4-4d3a-8926-56103918d30d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:17:23.396924 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:17:23.398977 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1249/2482] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:17:23 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:17:23.400412 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5d8a93-c3e4-4d3a-8926-56103918d30d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:17:23.402028 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5d8a93-c3e4-4d3a-8926-56103918d30d service swift] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:17:23.411096 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1234/2483] 10.222.0.82 () {60 vars in 1384 bytes} [Thu Aug 18 17:17:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 17:17:31.152031 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96885317-cde8-4573-9e2b-2ab06e5ac241 service neutron] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:17:31.154278 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-96885317-cde8-4573-9e2b-2ab06e5ac241 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:17:31.174569 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96885317-cde8-4573-9e2b-2ab06e5ac241 service neutron] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:17:31.181073 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec83560-adb7-4f81-88aa-e3739175b9c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55723c756cae4e7ab636054db48de02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:17:31.182680 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec83560-adb7-4f81-88aa-e3739175b9c1 service nova] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:17:31.183062 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec83560-adb7-4f81-88aa-e3739175b9c1 service nova] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:17:31.183344 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec83560-adb7-4f81-88aa-e3739175b9c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:17:31.185090 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-0ec83560-adb7-4f81-88aa-e3739175b9c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:17:31.258998 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec83560-adb7-4f81-88aa-e3739175b9c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:17:31.260960 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec83560-adb7-4f81-88aa-e3739175b9c1 service nova] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:17:31.274528 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1250/2484] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:17:31 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 17:17:33.435073 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=3 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:17:41.270711 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=81 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:17:47.390214 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5d8a93-c3e4-4d3a-8926-56103918d30d service swift] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:17:47.399813 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5d8a93-c3e4-4d3a-8926-56103918d30d service swift] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:17:47.403521 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fbc1dc-faff-4c61-890f-05ac8a66c384 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:17:47.404112 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fbc1dc-faff-4c61-890f-05ac8a66c384 service neutron] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:17:47.404394 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fbc1dc-faff-4c61-890f-05ac8a66c384 service neutron] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:17:47.404666 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fbc1dc-faff-4c61-890f-05ac8a66c384 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:17:47.406446 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-c8fbc1dc-faff-4c61-890f-05ac8a66c384 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:17:47.416466 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fbc1dc-faff-4c61-890f-05ac8a66c384 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:17:47.417912 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fbc1dc-faff-4c61-890f-05ac8a66c384 service neutron] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:17:47.426656 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1235/2485] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:17:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:17:54.210850 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec83560-adb7-4f81-88aa-e3739175b9c1 service nova] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:17:54.220527 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec83560-adb7-4f81-88aa-e3739175b9c1 service nova] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:17:54.224918 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbafa100-cb11-49f8-ade9-aebc9f2ad25b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:17:54.225552 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbafa100-cb11-49f8-ade9-aebc9f2ad25b service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:17:54.225833 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbafa100-cb11-49f8-ade9-aebc9f2ad25b service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:17:54.226106 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbafa100-cb11-49f8-ade9-aebc9f2ad25b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:17:54.227919 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-cbafa100-cb11-49f8-ade9-aebc9f2ad25b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:17:54.237964 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbafa100-cb11-49f8-ade9-aebc9f2ad25b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:17:54.239678 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbafa100-cb11-49f8-ade9-aebc9f2ad25b service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:17:54.248620 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1251/2486] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:17:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:17:54.655305 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5869d6-89f4-42b9-969c-d59778a62342 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:17:54.655828 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5869d6-89f4-42b9-969c-d59778a62342 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:17:54.655828 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5869d6-89f4-42b9-969c-d59778a62342 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:17:54.657226 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1236/2487] 10.222.0.82 () {54 vars in 921 bytes} [Thu Aug 18 17:17:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:17:54.665167 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb278af-eda3-4ee3-9e3c-4b50ad2298ec None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:17:54.665452 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb278af-eda3-4ee3-9e3c-4b50ad2298ec None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:17:54.665728 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb278af-eda3-4ee3-9e3c-4b50ad2298ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:17:54.685823 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-9fb278af-eda3-4ee3-9e3c-4b50ad2298ec None None] MFA Rules not processed for user `55723c756cae4e7ab636054db48de02d`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:17:54.687044 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-9fb278af-eda3-4ee3-9e3c-4b50ad2298ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:17:54.697402 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1252/2488] 10.222.0.82 () {58 vars in 1018 bytes} [Thu Aug 18 17:17:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:17:54.710286 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5869d6-89f4-42b9-969c-d59778a62342 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:17:54.712052 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-ea5869d6-89f4-42b9-969c-d59778a62342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:17:54.773875 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5869d6-89f4-42b9-969c-d59778a62342 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:17:54.777340 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fccf642-abad-419b-ab8c-920f188f64e8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bee1ba6bb324de4915904f83349b3c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:17:54.777939 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fccf642-abad-419b-ab8c-920f188f64e8 service glance] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:17:54.778218 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fccf642-abad-419b-ab8c-920f188f64e8 service glance] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:17:54.778488 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fccf642-abad-419b-ab8c-920f188f64e8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:17:54.789009 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fccf642-abad-419b-ab8c-920f188f64e8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:17:54.790494 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fccf642-abad-419b-ab8c-920f188f64e8 service glance] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:17:54.801443 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1237/2489] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:17:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:17:54.999606 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d646c-9dc7-41bb-9bd0-c2ca3117be04 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:17:54.999898 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d646c-9dc7-41bb-9bd0-c2ca3117be04 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:17:55.000182 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d646c-9dc7-41bb-9bd0-c2ca3117be04 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:17:55.001537 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1253/2490] 10.222.0.82 () {54 vars in 921 bytes} [Thu Aug 18 17:17:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:17:55.006363 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0c650b-52af-4092-b376-4c3a5a07543b None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:17:55.006684 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0c650b-52af-4092-b376-4c3a5a07543b None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:17:55.007006 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0c650b-52af-4092-b376-4c3a5a07543b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:17:55.026704 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-3d0c650b-52af-4092-b376-4c3a5a07543b None None] MFA Rules not processed for user `fe2403ede53148b2a113ffa2a605e987`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:17:55.027935 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-3d0c650b-52af-4092-b376-4c3a5a07543b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:17:55.034265 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1238/2491] 10.222.0.82 () {58 vars in 1018 bytes} [Thu Aug 18 17:17:55 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:17:55.046554 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652d646c-9dc7-41bb-9bd0-c2ca3117be04 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:17:55.048236 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-652d646c-9dc7-41bb-9bd0-c2ca3117be04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:17:55.061390 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652d646c-9dc7-41bb-9bd0-c2ca3117be04 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:17:55.064851 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec9ecbc-853d-41e0-ba79-befec73ae0dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:17:55.065462 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9ecbc-853d-41e0-ba79-befec73ae0dd service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:17:55.065739 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9ecbc-853d-41e0-ba79-befec73ae0dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:17:55.066015 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9ecbc-853d-41e0-ba79-befec73ae0dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:17:55.072778 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec9ecbc-853d-41e0-ba79-befec73ae0dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:17:55.074178 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec9ecbc-853d-41e0-ba79-befec73ae0dd service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:17:55.082365 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1254/2492] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:17:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:17:55.167390 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93fbb02-e8b8-4967-afff-72a7c2f43c7b None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:17:55.167681 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93fbb02-e8b8-4967-afff-72a7c2f43c7b None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:17:55.167964 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93fbb02-e8b8-4967-afff-72a7c2f43c7b None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:17:55.169622 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1239/2493] 10.222.0.82 () {54 vars in 921 bytes} [Thu Aug 18 17:17:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 17:17:55.175916 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622aa0d4-174b-482e-befd-c90ab34a7fd1 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:17:55.176209 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622aa0d4-174b-482e-befd-c90ab34a7fd1 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:17:55.176493 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622aa0d4-174b-482e-befd-c90ab34a7fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:17:55.197835 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-622aa0d4-174b-482e-befd-c90ab34a7fd1 None None] MFA Rules not processed for user `55723c756cae4e7ab636054db48de02d`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:17:55.199082 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-622aa0d4-174b-482e-befd-c90ab34a7fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:17:55.209908 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1255/2494] 10.222.0.82 () {58 vars in 1018 bytes} [Thu Aug 18 17:17:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:17:55.221330 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93fbb02-e8b8-4967-afff-72a7c2f43c7b None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:17:55.236482 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93fbb02-e8b8-4967-afff-72a7c2f43c7b None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:17:55.241355 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef72a7a-1da2-492f-908e-fc6435516e53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:17:55.241966 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef72a7a-1da2-492f-908e-fc6435516e53 service neutron] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:17:55.242251 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef72a7a-1da2-492f-908e-fc6435516e53 service neutron] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:17:55.242529 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef72a7a-1da2-492f-908e-fc6435516e53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:17:55.254087 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef72a7a-1da2-492f-908e-fc6435516e53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:17:55.256035 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef72a7a-1da2-492f-908e-fc6435516e53 service neutron] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:17:55.268357 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1240/2495] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:17:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 17:17:55.340014 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622aa0d4-174b-482e-befd-c90ab34a7fd1 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:17:55.341498 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-622aa0d4-174b-482e-befd-c90ab34a7fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:17:55.415444 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622aa0d4-174b-482e-befd-c90ab34a7fd1 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:17:55.421404 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdaa688-5e4e-401b-9dfb-4ee539358d16 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29653ba1a2a74232972ef7a9000518b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:17:55.422010 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdaa688-5e4e-401b-9dfb-4ee539358d16 service placement] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:17:55.422289 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdaa688-5e4e-401b-9dfb-4ee539358d16 service placement] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:17:55.422561 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdaa688-5e4e-401b-9dfb-4ee539358d16 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:17:55.424324 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-8bdaa688-5e4e-401b-9dfb-4ee539358d16 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:17:55.436444 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdaa688-5e4e-401b-9dfb-4ee539358d16 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:17:55.438093 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdaa688-5e4e-401b-9dfb-4ee539358d16 service placement] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:17:55.448863 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1256/2496] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:17:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:18:00.532374 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d95fbf-6708-4853-b148-4a162d6401c8 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:18:00.532848 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d95fbf-6708-4853-b148-4a162d6401c8 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:18:00.532945 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d95fbf-6708-4853-b148-4a162d6401c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:18:00.552866 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-26d95fbf-6708-4853-b148-4a162d6401c8 None None] MFA Rules not processed for user `5de19a54ed104bcdb35ee4e5e793f1e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:18:00.554305 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-26d95fbf-6708-4853-b148-4a162d6401c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:18:00.562600 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1241/2497] 10.222.0.82 () {58 vars in 1009 bytes} [Thu Aug 18 17:18:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:18:00.574384 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdaa688-5e4e-401b-9dfb-4ee539358d16 service placement] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:18:00.586135 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdaa688-5e4e-401b-9dfb-4ee539358d16 service placement] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:18:00.591868 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13551b1-69e3-43ed-b3ed-fcb903ddd151 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1381673f81584f4b8084823e3aa99e2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:18:00.592496 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13551b1-69e3-43ed-b3ed-fcb903ddd151 service swift] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:18:00.592820 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13551b1-69e3-43ed-b3ed-fcb903ddd151 service swift] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:18:00.593211 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13551b1-69e3-43ed-b3ed-fcb903ddd151 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:18:00.602859 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13551b1-69e3-43ed-b3ed-fcb903ddd151 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:18:00.604550 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13551b1-69e3-43ed-b3ed-fcb903ddd151 service swift] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:18:00.612824 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1257/2498] 10.222.0.82 () {60 vars in 1384 bytes} [Thu Aug 18 17:18:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 17:18:10.547297 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=44 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:18:10.609605 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=56 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:18:32.306283 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d95fbf-6708-4853-b148-4a162d6401c8 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:18:32.321614 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d95fbf-6708-4853-b148-4a162d6401c8 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:18:32.327322 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e20a70-2cb7-4487-88ce-9a0fd454384d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55723c756cae4e7ab636054db48de02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:18:32.328083 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e20a70-2cb7-4487-88ce-9a0fd454384d service nova] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:18:32.328369 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e20a70-2cb7-4487-88ce-9a0fd454384d service nova] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:18:32.328647 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e20a70-2cb7-4487-88ce-9a0fd454384d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:18:32.330389 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-e7e20a70-2cb7-4487-88ce-9a0fd454384d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:18:32.346361 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e20a70-2cb7-4487-88ce-9a0fd454384d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:18:32.348050 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e20a70-2cb7-4487-88ce-9a0fd454384d service nova] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:18:32.359735 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1242/2499] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:18:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:18:35.540923 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0341f40-c808-46a2-a940-ec8a2a36a0e9 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:18:35.541436 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0341f40-c808-46a2-a940-ec8a2a36a0e9 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:18:35.541541 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0341f40-c808-46a2-a940-ec8a2a36a0e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:18:35.561740 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-d0341f40-c808-46a2-a940-ec8a2a36a0e9 None None] MFA Rules not processed for user `5de19a54ed104bcdb35ee4e5e793f1e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:18:35.563157 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-d0341f40-c808-46a2-a940-ec8a2a36a0e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:18:35.570278 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1258/2500] 10.222.0.82 () {58 vars in 967 bytes} [Thu Aug 18 17:18:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:18:35.592499 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e20a70-2cb7-4487-88ce-9a0fd454384d service nova] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:18:35.604501 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e20a70-2cb7-4487-88ce-9a0fd454384d service nova] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:18:35.609513 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c493911b-e871-4d40-9fac-014703a37626 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1381673f81584f4b8084823e3aa99e2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:18:35.610101 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c493911b-e871-4d40-9fac-014703a37626 service swift] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:18:35.610385 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c493911b-e871-4d40-9fac-014703a37626 service swift] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:18:35.610659 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c493911b-e871-4d40-9fac-014703a37626 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:18:35.618563 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c493911b-e871-4d40-9fac-014703a37626 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:18:35.620185 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c493911b-e871-4d40-9fac-014703a37626 service swift] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:18:35.628206 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1243/2501] 10.222.0.82 () {60 vars in 1384 bytes} [Thu Aug 18 17:18:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 17:18:39.270791 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153d747-f763-43b2-a8cb-b416435c374f None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:18:39.271450 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153d747-f763-43b2-a8cb-b416435c374f None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:18:39.271450 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153d747-f763-43b2-a8cb-b416435c374f None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:18:39.272840 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1259/2502] 10.222.0.82 () {54 vars in 921 bytes} [Thu Aug 18 17:18:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:18:39.289372 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b01f24e-c7a5-4b06-bd99-3c6809038559 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:18:39.289890 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b01f24e-c7a5-4b06-bd99-3c6809038559 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:18:39.290088 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b01f24e-c7a5-4b06-bd99-3c6809038559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:18:39.310581 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-9b01f24e-c7a5-4b06-bd99-3c6809038559 None None] MFA Rules not processed for user `29653ba1a2a74232972ef7a9000518b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:18:39.311963 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-9b01f24e-c7a5-4b06-bd99-3c6809038559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:18:39.321331 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1244/2503] 10.222.0.82 () {58 vars in 1018 bytes} [Thu Aug 18 17:18:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:18:39.337298 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8153d747-f763-43b2-a8cb-b416435c374f None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:18:39.339034 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-8153d747-f763-43b2-a8cb-b416435c374f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:18:39.355875 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8153d747-f763-43b2-a8cb-b416435c374f None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:18:39.361822 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739986c8-327e-4e5a-91fd-e48fdd8cef63 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29653ba1a2a74232972ef7a9000518b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:18:39.362415 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739986c8-327e-4e5a-91fd-e48fdd8cef63 service placement] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:18:39.362697 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739986c8-327e-4e5a-91fd-e48fdd8cef63 service placement] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:18:39.363002 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739986c8-327e-4e5a-91fd-e48fdd8cef63 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:18:39.371465 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739986c8-327e-4e5a-91fd-e48fdd8cef63 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:18:39.373087 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739986c8-327e-4e5a-91fd-e48fdd8cef63 service placement] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:18:39.383773 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1260/2504] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:18:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:18:39.409253 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b01f24e-c7a5-4b06-bd99-3c6809038559 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:18:39.411003 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-9b01f24e-c7a5-4b06-bd99-3c6809038559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:18:39.480320 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b01f24e-c7a5-4b06-bd99-3c6809038559 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:18:39.485550 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0377f26-1a6c-48ab-893f-9e2de9d9f1c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:18:39.486236 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0377f26-1a6c-48ab-893f-9e2de9d9f1c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:18:39.486517 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0377f26-1a6c-48ab-893f-9e2de9d9f1c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:18:39.486788 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0377f26-1a6c-48ab-893f-9e2de9d9f1c7 admin admin] PATH_INFO: `/v3/users/32f58f87972d46be9496e6216f1f5ca1` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:18:39.487990 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0377f26-1a6c-48ab-893f-9e2de9d9f1c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32f58f87972d46be9496e6216f1f5ca1)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:18:39.489913 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0377f26-1a6c-48ab-893f-9e2de9d9f1c7 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:18:39.676727 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-d0377f26-1a6c-48ab-893f-9e2de9d9f1c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32f58f87972d46be9496e6216f1f5ca1'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:18:39.689119 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-d0377f26-1a6c-48ab-893f-9e2de9d9f1c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32f58f87972d46be9496e6216f1f5ca1'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:18:39.691294 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-d0377f26-1a6c-48ab-893f-9e2de9d9f1c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32f58f87972d46be9496e6216f1f5ca1'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:18:39.767573 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-d0377f26-1a6c-48ab-893f-9e2de9d9f1c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32f58f87972d46be9496e6216f1f5ca1'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:18:39.768221 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-d0377f26-1a6c-48ab-893f-9e2de9d9f1c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32f58f87972d46be9496e6216f1f5ca1'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:18:39.781103 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1245/2505] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:18:39 2022] DELETE /identity/v3/users/32f58f87972d46be9496e6216f1f5ca1 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:18:40.025715 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739986c8-327e-4e5a-91fd-e48fdd8cef63 service placement] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:18:40.037079 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739986c8-327e-4e5a-91fd-e48fdd8cef63 service placement] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:18:40.094735 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f78e10-02d2-4614-8572-f500cb7f1a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:18:40.095411 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f78e10-02d2-4614-8572-f500cb7f1a4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:18:40.095692 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f78e10-02d2-4614-8572-f500cb7f1a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:18:40.095966 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f78e10-02d2-4614-8572-f500cb7f1a4c admin admin] PATH_INFO: `/v3/projects/420292ae9c0547cc9755f24c23d419c3` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:18:40.097105 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f78e10-02d2-4614-8572-f500cb7f1a4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=420292ae9c0547cc9755f24c23d419c3)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:18:40.099027 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f78e10-02d2-4614-8572-f500cb7f1a4c admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:18:40.148202 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0377f26-1a6c-48ab-893f-9e2de9d9f1c7 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:18:40.149645 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-d0377f26-1a6c-48ab-893f-9e2de9d9f1c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:18:40.162695 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0377f26-1a6c-48ab-893f-9e2de9d9f1c7 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:18:40.167210 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6860d411-8b8a-4c4b-af85-09c15160a2f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:18:40.167829 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6860d411-8b8a-4c4b-af85-09c15160a2f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:18:40.168110 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6860d411-8b8a-4c4b-af85-09c15160a2f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:18:40.168384 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6860d411-8b8a-4c4b-af85-09c15160a2f4 admin admin] PATH_INFO: `/v3/users/bf9c01c6d32a493884eb279115c9078a` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:18:40.169538 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6860d411-8b8a-4c4b-af85-09c15160a2f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf9c01c6d32a493884eb279115c9078a)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:18:40.171501 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6860d411-8b8a-4c4b-af85-09c15160a2f4 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:18:40.196198 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-38f78e10-02d2-4614-8572-f500cb7f1a4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '420292ae9c0547cc9755f24c23d419c3', 'request_id': 'req-38f78e10-02d2-4614-8572-f500cb7f1a4c'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:18:40.202472 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-38f78e10-02d2-4614-8572-f500cb7f1a4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '420292ae9c0547cc9755f24c23d419c3', 'request_id': 'req-38f78e10-02d2-4614-8572-f500cb7f1a4c'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:18:40.291057 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-38f78e10-02d2-4614-8572-f500cb7f1a4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '420292ae9c0547cc9755f24c23d419c3', 'request_id': 'req-38f78e10-02d2-4614-8572-f500cb7f1a4c'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:18:40.303364 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-38f78e10-02d2-4614-8572-f500cb7f1a4c admin admin] The token cache is being invalidate because project 420292ae9c0547cc9755f24c23d419c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:18:40.303657 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-38f78e10-02d2-4614-8572-f500cb7f1a4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:18:40.304310 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-38f78e10-02d2-4614-8572-f500cb7f1a4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:18:40.305808 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1261/2506] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:18:40 2022] DELETE /identity/v3/projects/420292ae9c0547cc9755f24c23d419c3 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:18:40.310812 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5105fd75-4149-45b8-b251-5d579e6e65d2 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:18:40.311142 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5105fd75-4149-45b8-b251-5d579e6e65d2 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:18:40.311447 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5105fd75-4149-45b8-b251-5d579e6e65d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:18:40.332602 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-5105fd75-4149-45b8-b251-5d579e6e65d2 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:18:40.333824 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-5105fd75-4149-45b8-b251-5d579e6e65d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:18:40.364714 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:18:40.368949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-6860d411-8b8a-4c4b-af85-09c15160a2f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf9c01c6d32a493884eb279115c9078a'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:18:40.381372 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-6860d411-8b8a-4c4b-af85-09c15160a2f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf9c01c6d32a493884eb279115c9078a'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:18:40.383231 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-6860d411-8b8a-4c4b-af85-09c15160a2f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf9c01c6d32a493884eb279115c9078a'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:18:40.396842 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1262/2507] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:18:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 17:18:40.404688 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929aa01f-8517-48b1-9ba5-c87d4b27a634 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:18:40.404973 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929aa01f-8517-48b1-9ba5-c87d4b27a634 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:18:40.405245 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929aa01f-8517-48b1-9ba5-c87d4b27a634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:18:40.429504 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-929aa01f-8517-48b1-9ba5-c87d4b27a634 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:18:40.430712 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-929aa01f-8517-48b1-9ba5-c87d4b27a634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:18:40.440913 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1263/2508] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:18:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 17:18:40.445097 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929aa01f-8517-48b1-9ba5-c87d4b27a634 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:18:40.451085 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:18:40.455495 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929aa01f-8517-48b1-9ba5-c87d4b27a634 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:18:40.461181 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df25a158-5fbe-4e54-b108-2373d6b1bfca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:18:40.461731 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25a158-5fbe-4e54-b108-2373d6b1bfca admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:18:40.462009 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25a158-5fbe-4e54-b108-2373d6b1bfca admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:18:40.462276 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25a158-5fbe-4e54-b108-2373d6b1bfca admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:18:40.463459 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df25a158-5fbe-4e54-b108-2373d6b1bfca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:18:40.463740 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-6860d411-8b8a-4c4b-af85-09c15160a2f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf9c01c6d32a493884eb279115c9078a'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:18:40.463961 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df25a158-5fbe-4e54-b108-2373d6b1bfca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:18:40.464504 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-6860d411-8b8a-4c4b-af85-09c15160a2f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf9c01c6d32a493884eb279115c9078a'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:18:40.466018 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df25a158-5fbe-4e54-b108-2373d6b1bfca admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:18:40.479715 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1246/2509] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:18:40 2022] DELETE /identity/v3/users/bf9c01c6d32a493884eb279115c9078a => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:18:40.479967 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1264/2510] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:18:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 17:18:40.485270 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df25a158-5fbe-4e54-b108-2373d6b1bfca admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:18:40.485563 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d38de0-da6d-41bc-9210-6aa333c4dc5c None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:18:40.486162 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d38de0-da6d-41bc-9210-6aa333c4dc5c None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:18:40.486560 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d38de0-da6d-41bc-9210-6aa333c4dc5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:18:40.487251 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-df25a158-5fbe-4e54-b108-2373d6b1bfca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:18:40.500183 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df25a158-5fbe-4e54-b108-2373d6b1bfca admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:18:40.506396 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-62d38de0-da6d-41bc-9210-6aa333c4dc5c None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:18:40.507726 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-62d38de0-da6d-41bc-9210-6aa333c4dc5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:18:40.564229 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65124283-0d02-4dd5-9f37-e7483cac8798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:18:40.564767 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65124283-0d02-4dd5-9f37-e7483cac8798 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:18:40.565054 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65124283-0d02-4dd5-9f37-e7483cac8798 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:18:40.565365 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65124283-0d02-4dd5-9f37-e7483cac8798 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:18:40.566000 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65124283-0d02-4dd5-9f37-e7483cac8798 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:18:40.567965 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65124283-0d02-4dd5-9f37-e7483cac8798 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:18:40.572268 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1247/2511] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:18:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 17:18:40.583076 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d38de0-da6d-41bc-9210-6aa333c4dc5c None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:18:40.584540 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-62d38de0-da6d-41bc-9210-6aa333c4dc5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:18:40.645681 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d38de0-da6d-41bc-9210-6aa333c4dc5c None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:18:40.649443 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2504e297-94a9-4f1b-9735-83554b814446 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:18:40.650059 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2504e297-94a9-4f1b-9735-83554b814446 service neutron] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:18:40.650345 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2504e297-94a9-4f1b-9735-83554b814446 service neutron] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:18:40.650643 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2504e297-94a9-4f1b-9735-83554b814446 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:18:40.660747 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1265/2512] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:18:40 2022] POST /identity/v3/projects => generated 391 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:18:40.661544 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2504e297-94a9-4f1b-9735-83554b814446 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:18:40.662505 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65124283-0d02-4dd5-9f37-e7483cac8798 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:18:40.662758 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2504e297-94a9-4f1b-9735-83554b814446 service neutron] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:18:40.674127 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1248/2513] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:18:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 17:18:40.733642 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65124283-0d02-4dd5-9f37-e7483cac8798 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:18:40.739577 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6272d139-fc35-4582-91bb-be708e84336d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:18:40.740335 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6272d139-fc35-4582-91bb-be708e84336d admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:18:40.740619 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6272d139-fc35-4582-91bb-be708e84336d admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:18:40.740893 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6272d139-fc35-4582-91bb-be708e84336d admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:18:40.741578 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6272d139-fc35-4582-91bb-be708e84336d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:18:40.743725 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6272d139-fc35-4582-91bb-be708e84336d admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:18:40.861276 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1266/2514] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:18:40 2022] POST /identity/v3/users => generated 296 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:18:40.877804 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2504e297-94a9-4f1b-9735-83554b814446 service neutron] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:18:40.882326 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2504e297-94a9-4f1b-9735-83554b814446 service neutron] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:18:40.887180 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659e501a-2b97-409f-b8ff-5e8cdb539e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:18:40.887789 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659e501a-2b97-409f-b8ff-5e8cdb539e51 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:18:40.888100 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659e501a-2b97-409f-b8ff-5e8cdb539e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:18:40.888487 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659e501a-2b97-409f-b8ff-5e8cdb539e51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:18:40.889142 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659e501a-2b97-409f-b8ff-5e8cdb539e51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:18:40.891666 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659e501a-2b97-409f-b8ff-5e8cdb539e51 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:18:40.908141 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1249/2515] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:18:40 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 17:18:40.910436 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6272d139-fc35-4582-91bb-be708e84336d admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:18:40.931155 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6272d139-fc35-4582-91bb-be708e84336d admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:18:40.937456 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaac905-251c-46bf-ba93-0afbf381b473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:18:40.938221 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaac905-251c-46bf-ba93-0afbf381b473 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:18:40.938507 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaac905-251c-46bf-ba93-0afbf381b473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:18:40.938786 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaac905-251c-46bf-ba93-0afbf381b473 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:18:40.939475 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaac905-251c-46bf-ba93-0afbf381b473 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:18:40.941123 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaac905-251c-46bf-ba93-0afbf381b473 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:18:40.955441 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1267/2516] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:18:40 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:18:40.960288 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659e501a-2b97-409f-b8ff-5e8cdb539e51 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:18:40.971700 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659e501a-2b97-409f-b8ff-5e8cdb539e51 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:18:40.977079 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb6759a-fa76-4cca-bb53-037543a46349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:18:40.977752 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb6759a-fa76-4cca-bb53-037543a46349 admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:18:40.978037 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb6759a-fa76-4cca-bb53-037543a46349 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:18:40.978313 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb6759a-fa76-4cca-bb53-037543a46349 admin admin] PATH_INFO: `/v3/projects/e9b9a1097b7c4d269ac1ee8cd3de0048/users/b1a3a469b1504f8bb9e97649b2950f48/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:18:41.002196 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb6759a-fa76-4cca-bb53-037543a46349 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9b9a1097b7c4d269ac1ee8cd3de0048, user_id=b1a3a469b1504f8bb9e97649b2950f48, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:18:41.002196 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb6759a-fa76-4cca-bb53-037543a46349 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:18:41.071529 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1250/2517] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:18:40 2022] PUT /identity/v3/projects/e9b9a1097b7c4d269ac1ee8cd3de0048/users/b1a3a469b1504f8bb9e97649b2950f48/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:18:41.142506 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b19eac-aa44-41c9-aa45-db89e0a742a1 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:18:41.142717 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b19eac-aa44-41c9-aa45-db89e0a742a1 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:18:41.143029 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b19eac-aa44-41c9-aa45-db89e0a742a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:18:41.145184 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb6759a-fa76-4cca-bb53-037543a46349 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:18:41.147263 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-8eb6759a-fa76-4cca-bb53-037543a46349 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:18:41.183966 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-b3b19eac-aa44-41c9-aa45-db89e0a742a1 None None] MFA Rules not processed for user `b1a3a469b1504f8bb9e97649b2950f48`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:18:41.184988 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:18:41.186065 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-b3b19eac-aa44-41c9-aa45-db89e0a742a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:18:41.221311 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb6759a-fa76-4cca-bb53-037543a46349 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:18:41.226200 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b0edc5-74e1-49a2-96d1-84d2a8ce8c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:18:41.227560 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b0edc5-74e1-49a2-96d1-84d2a8ce8c3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:18:41.227852 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b0edc5-74e1-49a2-96d1-84d2a8ce8c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:18:41.228342 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b0edc5-74e1-49a2-96d1-84d2a8ce8c3d admin admin] PATH_INFO: `/v3/projects/da53e08509254c14b51831321cc7dc5f` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:18:41.229533 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b0edc5-74e1-49a2-96d1-84d2a8ce8c3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=da53e08509254c14b51831321cc7dc5f)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:18:41.231782 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b0edc5-74e1-49a2-96d1-84d2a8ce8c3d admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:18:41.238705 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:18:41.259706 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1268/2518] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:18:41 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:18:41.271312 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b19eac-aa44-41c9-aa45-db89e0a742a1 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:18:41.272647 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-b3b19eac-aa44-41c9-aa45-db89e0a742a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:18:41.329073 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b19eac-aa44-41c9-aa45-db89e0a742a1 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:18:41.332923 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf20a77-5734-4aed-b42e-241025b3ccd3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:18:41.333543 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf20a77-5734-4aed-b42e-241025b3ccd3 service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:18:41.333834 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf20a77-5734-4aed-b42e-241025b3ccd3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:18:41.334129 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf20a77-5734-4aed-b42e-241025b3ccd3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:18:41.342728 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf20a77-5734-4aed-b42e-241025b3ccd3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:18:41.343407 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-68b0edc5-74e1-49a2-96d1-84d2a8ce8c3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da53e08509254c14b51831321cc7dc5f', 'request_id': 'req-68b0edc5-74e1-49a2-96d1-84d2a8ce8c3d'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:18:41.344357 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf20a77-5734-4aed-b42e-241025b3ccd3 service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:18:41.353342 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1269/2519] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:18:41 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 17:18:41.462783 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-68b0edc5-74e1-49a2-96d1-84d2a8ce8c3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da53e08509254c14b51831321cc7dc5f', 'request_id': 'req-68b0edc5-74e1-49a2-96d1-84d2a8ce8c3d'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:18:41.469570 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-68b0edc5-74e1-49a2-96d1-84d2a8ce8c3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da53e08509254c14b51831321cc7dc5f', 'request_id': 'req-68b0edc5-74e1-49a2-96d1-84d2a8ce8c3d'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:18:41.481658 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-68b0edc5-74e1-49a2-96d1-84d2a8ce8c3d admin admin] The token cache is being invalidate because project da53e08509254c14b51831321cc7dc5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81650) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:18:41.482002 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-68b0edc5-74e1-49a2-96d1-84d2a8ce8c3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:18:41.482678 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-68b0edc5-74e1-49a2-96d1-84d2a8ce8c3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:18:41.484326 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1251/2520] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:18:41 2022] DELETE /identity/v3/projects/da53e08509254c14b51831321cc7dc5f => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:18:51.560440 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=33 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:18:51.617508 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=39,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:18:53.277907 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf20a77-5734-4aed-b42e-241025b3ccd3 service cinder] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:18:53.277907 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-dcf20a77-5734-4aed-b42e-241025b3ccd3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:18:53.353499 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf20a77-5734-4aed-b42e-241025b3ccd3 service cinder] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:18:53.417121 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b673b4-2bbf-47af-8434-e334c0b4107b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55723c756cae4e7ab636054db48de02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:18:53.417724 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b673b4-2bbf-47af-8434-e334c0b4107b service nova] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:18:53.418002 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b673b4-2bbf-47af-8434-e334c0b4107b service nova] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:18:53.418273 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b673b4-2bbf-47af-8434-e334c0b4107b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:18:53.420030 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-19b673b4-2bbf-47af-8434-e334c0b4107b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:18:53.477046 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b673b4-2bbf-47af-8434-e334c0b4107b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:18:53.478734 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b673b4-2bbf-47af-8434-e334c0b4107b service nova] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:18:53.487692 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1270/2521] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:18:53 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:19:03.483473 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=84 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:19:08.468362 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b0edc5-74e1-49a2-96d1-84d2a8ce8c3d admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:19:08.469774 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-68b0edc5-74e1-49a2-96d1-84d2a8ce8c3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:19:08.534135 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b0edc5-74e1-49a2-96d1-84d2a8ce8c3d admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:19:08.539324 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783e2155-2170-4fe1-8ff6-2e32c9d442c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:19:08.539947 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e2155-2170-4fe1-8ff6-2e32c9d442c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:19:08.540226 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e2155-2170-4fe1-8ff6-2e32c9d442c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:19:08.540499 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e2155-2170-4fe1-8ff6-2e32c9d442c5 admin admin] PATH_INFO: `/v3/users/b1a3a469b1504f8bb9e97649b2950f48` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:19:08.541595 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783e2155-2170-4fe1-8ff6-2e32c9d442c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1a3a469b1504f8bb9e97649b2950f48)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:19:08.543557 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783e2155-2170-4fe1-8ff6-2e32c9d442c5 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:19:08.769760 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-783e2155-2170-4fe1-8ff6-2e32c9d442c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1a3a469b1504f8bb9e97649b2950f48'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:19:08.782698 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-783e2155-2170-4fe1-8ff6-2e32c9d442c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1a3a469b1504f8bb9e97649b2950f48'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:19:08.783441 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-783e2155-2170-4fe1-8ff6-2e32c9d442c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1a3a469b1504f8bb9e97649b2950f48'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:19:08.847595 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-783e2155-2170-4fe1-8ff6-2e32c9d442c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1a3a469b1504f8bb9e97649b2950f48'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:19:08.848153 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-783e2155-2170-4fe1-8ff6-2e32c9d442c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1a3a469b1504f8bb9e97649b2950f48'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:19:08.860790 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1252/2522] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:19:08 2022] DELETE /identity/v3/users/b1a3a469b1504f8bb9e97649b2950f48 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:19:08.865625 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a3c361-c40e-4d65-8300-b73ae13af412 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:19:08.866167 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a3c361-c40e-4d65-8300-b73ae13af412 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:19:08.866330 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a3c361-c40e-4d65-8300-b73ae13af412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:19:08.886991 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-82a3c361-c40e-4d65-8300-b73ae13af412 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:19:08.888203 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-82a3c361-c40e-4d65-8300-b73ae13af412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:19:08.958533 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1271/2523] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:19:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:19:08.969850 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783e2155-2170-4fe1-8ff6-2e32c9d442c5 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:19:08.971396 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-783e2155-2170-4fe1-8ff6-2e32c9d442c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:19:09.030959 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783e2155-2170-4fe1-8ff6-2e32c9d442c5 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:19:09.034510 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c946a0-7adc-4cd6-bbfd-323258edf2f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:19:09.035159 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c946a0-7adc-4cd6-bbfd-323258edf2f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:19:09.035448 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c946a0-7adc-4cd6-bbfd-323258edf2f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:19:09.035723 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c946a0-7adc-4cd6-bbfd-323258edf2f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:19:09.044684 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c946a0-7adc-4cd6-bbfd-323258edf2f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:19:09.046129 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c946a0-7adc-4cd6-bbfd-323258edf2f6 service neutron] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:19:09.055577 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1253/2524] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:19:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:19:09.439362 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a3c361-c40e-4d65-8300-b73ae13af412 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:19:09.440813 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-82a3c361-c40e-4d65-8300-b73ae13af412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:19:09.453841 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a3c361-c40e-4d65-8300-b73ae13af412 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:19:09.458713 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e3caed-231f-4309-86bd-ab5a530a267a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:19:09.459330 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e3caed-231f-4309-86bd-ab5a530a267a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:19:09.459649 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e3caed-231f-4309-86bd-ab5a530a267a admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:19:09.459944 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e3caed-231f-4309-86bd-ab5a530a267a admin admin] PATH_INFO: `/v3/projects/e9b9a1097b7c4d269ac1ee8cd3de0048` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:19:09.461108 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e3caed-231f-4309-86bd-ab5a530a267a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9b9a1097b7c4d269ac1ee8cd3de0048)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:19:09.463043 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e3caed-231f-4309-86bd-ab5a530a267a admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:19:09.572946 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-b5e3caed-231f-4309-86bd-ab5a530a267a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9b9a1097b7c4d269ac1ee8cd3de0048', 'request_id': 'req-b5e3caed-231f-4309-86bd-ab5a530a267a'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:19:09.579092 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-b5e3caed-231f-4309-86bd-ab5a530a267a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9b9a1097b7c4d269ac1ee8cd3de0048', 'request_id': 'req-b5e3caed-231f-4309-86bd-ab5a530a267a'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:19:09.647359 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-b5e3caed-231f-4309-86bd-ab5a530a267a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9b9a1097b7c4d269ac1ee8cd3de0048', 'request_id': 'req-b5e3caed-231f-4309-86bd-ab5a530a267a'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:19:09.657719 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-b5e3caed-231f-4309-86bd-ab5a530a267a admin admin] The token cache is being invalidate because project e9b9a1097b7c4d269ac1ee8cd3de0048 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:19:09.658004 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-b5e3caed-231f-4309-86bd-ab5a530a267a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:19:09.658624 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-b5e3caed-231f-4309-86bd-ab5a530a267a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:19:09.660160 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1272/2525] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:19:09 2022] DELETE /identity/v3/projects/e9b9a1097b7c4d269ac1ee8cd3de0048 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:19:19.051334 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:19:19.654791 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:19:24.443144 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82cf7cb-95e2-4ca3-a7ba-ff6d71cb3806 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:19:24.443657 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82cf7cb-95e2-4ca3-a7ba-ff6d71cb3806 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:19:24.443761 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82cf7cb-95e2-4ca3-a7ba-ff6d71cb3806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:19:24.463514 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-c82cf7cb-95e2-4ca3-a7ba-ff6d71cb3806 None None] MFA Rules not processed for user `ecdea0572cc64ca394cd4d977f536d5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:19:24.464731 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-c82cf7cb-95e2-4ca3-a7ba-ff6d71cb3806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:19:24.520491 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1254/2526] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:19:24 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:19:30.968695 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e3caed-231f-4309-86bd-ab5a530a267a admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:19:30.970367 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-b5e3caed-231f-4309-86bd-ab5a530a267a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:19:31.048479 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e3caed-231f-4309-86bd-ab5a530a267a admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:19:31.054068 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc798d94-4aa0-496d-b190-eab2b9d76afd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55723c756cae4e7ab636054db48de02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:19:31.054704 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc798d94-4aa0-496d-b190-eab2b9d76afd service nova] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:19:31.055050 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc798d94-4aa0-496d-b190-eab2b9d76afd service nova] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:19:31.055337 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc798d94-4aa0-496d-b190-eab2b9d76afd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:19:31.057308 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-cc798d94-4aa0-496d-b190-eab2b9d76afd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:19:31.069823 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc798d94-4aa0-496d-b190-eab2b9d76afd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:19:31.071449 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc798d94-4aa0-496d-b190-eab2b9d76afd service nova] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:19:31.081963 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1273/2527] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:19:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:19:34.513932 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=25 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:19:41.077624 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=34 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:20:00.523606 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82cf7cb-95e2-4ca3-a7ba-ff6d71cb3806 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:20:00.525089 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-c82cf7cb-95e2-4ca3-a7ba-ff6d71cb3806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:20:00.541282 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82cf7cb-95e2-4ca3-a7ba-ff6d71cb3806 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:20:00.558981 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4a89af-383f-4321-87c8-d43f48104d9d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55723c756cae4e7ab636054db48de02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:20:00.558981 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a89af-383f-4321-87c8-d43f48104d9d service nova] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:20:00.558981 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a89af-383f-4321-87c8-d43f48104d9d service nova] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:20:00.558981 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a89af-383f-4321-87c8-d43f48104d9d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:20:00.559623 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-7b4a89af-383f-4321-87c8-d43f48104d9d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:20:00.565081 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4a89af-383f-4321-87c8-d43f48104d9d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:20:00.566823 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4a89af-383f-4321-87c8-d43f48104d9d service nova] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:20:00.578640 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1255/2528] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:20:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:20:05.482839 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b1d5a3-ba66-4c94-a85a-f1cee08176f5 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:20:05.483389 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b1d5a3-ba66-4c94-a85a-f1cee08176f5 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:20:05.484020 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b1d5a3-ba66-4c94-a85a-f1cee08176f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:20:05.502994 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-77b1d5a3-ba66-4c94-a85a-f1cee08176f5 None None] MFA Rules not processed for user `22e737cba90b4d8890ecd692e6456470`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:20:05.504131 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-77b1d5a3-ba66-4c94-a85a-f1cee08176f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:20:05.558257 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1274/2529] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:20:05 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:20:05.861153 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4a89af-383f-4321-87c8-d43f48104d9d service nova] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:20:05.862574 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-7b4a89af-383f-4321-87c8-d43f48104d9d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:20:05.918155 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4a89af-383f-4321-87c8-d43f48104d9d service nova] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:20:05.921994 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2208047-fc64-40a5-8fb9-887dc350b921 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:20:05.922589 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2208047-fc64-40a5-8fb9-887dc350b921 service neutron] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:20:05.922871 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2208047-fc64-40a5-8fb9-887dc350b921 service neutron] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:20:05.923176 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2208047-fc64-40a5-8fb9-887dc350b921 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:20:05.925184 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-e2208047-fc64-40a5-8fb9-887dc350b921 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:20:05.934367 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2208047-fc64-40a5-8fb9-887dc350b921 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:20:05.935829 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2208047-fc64-40a5-8fb9-887dc350b921 service neutron] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:20:05.945234 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1256/2530] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:20:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:20:15.551675 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=25 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:20:15.941031 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=32 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:20:33.977752 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b1d5a3-ba66-4c94-a85a-f1cee08176f5 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:20:33.979183 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-77b1d5a3-ba66-4c94-a85a-f1cee08176f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:20:34.129377 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b1d5a3-ba66-4c94-a85a-f1cee08176f5 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:20:34.134491 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94cf603-2457-451c-8858-258ce0b84a78 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29653ba1a2a74232972ef7a9000518b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:20:34.135137 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94cf603-2457-451c-8858-258ce0b84a78 service placement] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:20:34.135416 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94cf603-2457-451c-8858-258ce0b84a78 service placement] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:20:34.135690 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94cf603-2457-451c-8858-258ce0b84a78 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:20:34.137450 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-f94cf603-2457-451c-8858-258ce0b84a78 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:20:34.150014 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94cf603-2457-451c-8858-258ce0b84a78 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:20:34.152989 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94cf603-2457-451c-8858-258ce0b84a78 service placement] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:20:34.162791 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1275/2531] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:20:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:20:35.988990 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2208047-fc64-40a5-8fb9-887dc350b921 service neutron] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:20:36.035455 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2208047-fc64-40a5-8fb9-887dc350b921 service neutron] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:20:36.041673 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66641c68-9917-47f2-a179-5f91e7931f07 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55723c756cae4e7ab636054db48de02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:20:36.042266 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66641c68-9917-47f2-a179-5f91e7931f07 service nova] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:20:36.042546 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66641c68-9917-47f2-a179-5f91e7931f07 service nova] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:20:36.042820 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66641c68-9917-47f2-a179-5f91e7931f07 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:20:36.044531 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-66641c68-9917-47f2-a179-5f91e7931f07 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:20:36.102747 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66641c68-9917-47f2-a179-5f91e7931f07 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:20:36.104373 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66641c68-9917-47f2-a179-5f91e7931f07 service nova] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:20:36.113180 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1257/2532] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:20:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:20:44.157609 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=63 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:20:45.955258 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94cf603-2457-451c-8858-258ce0b84a78 service placement] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:20:45.956746 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-f94cf603-2457-451c-8858-258ce0b84a78 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:20:45.990741 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94cf603-2457-451c-8858-258ce0b84a78 service placement] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:20:45.994310 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e41ea7b-8afd-4287-a95d-d2275363a323 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:20:45.994927 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e41ea7b-8afd-4287-a95d-d2275363a323 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:20:45.995222 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e41ea7b-8afd-4287-a95d-d2275363a323 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:20:45.995482 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e41ea7b-8afd-4287-a95d-d2275363a323 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:20:45.997146 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-6e41ea7b-8afd-4287-a95d-d2275363a323 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:20:46.109103 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=43 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:20:46.212049 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e41ea7b-8afd-4287-a95d-d2275363a323 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:20:46.213499 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e41ea7b-8afd-4287-a95d-d2275363a323 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:20:46.232565 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1276/2533] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:20:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:20:48.650335 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66641c68-9917-47f2-a179-5f91e7931f07 service nova] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:20:48.662178 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66641c68-9917-47f2-a179-5f91e7931f07 service nova] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:20:48.667695 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed5e379-4866-4c3f-9e63-ce89fe2f6963 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55723c756cae4e7ab636054db48de02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:20:48.668313 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed5e379-4866-4c3f-9e63-ce89fe2f6963 service nova] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:20:48.668598 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed5e379-4866-4c3f-9e63-ce89fe2f6963 service nova] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:20:48.668876 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed5e379-4866-4c3f-9e63-ce89fe2f6963 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:20:48.670586 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-1ed5e379-4866-4c3f-9e63-ce89fe2f6963 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:20:48.734600 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed5e379-4866-4c3f-9e63-ce89fe2f6963 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:20:48.736255 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed5e379-4866-4c3f-9e63-ce89fe2f6963 service nova] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:20:48.746599 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1258/2534] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:20:48 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:20:49.718680 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e41ea7b-8afd-4287-a95d-d2275363a323 service neutron] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:20:49.720139 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-6e41ea7b-8afd-4287-a95d-d2275363a323 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:20:49.737946 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e41ea7b-8afd-4287-a95d-d2275363a323 service neutron] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:20:49.751417 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0df024-9e29-4410-83f1-ebbfc585b24e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29653ba1a2a74232972ef7a9000518b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:20:49.752077 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0df024-9e29-4410-83f1-ebbfc585b24e service placement] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:20:49.752358 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0df024-9e29-4410-83f1-ebbfc585b24e service placement] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:20:49.753165 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0df024-9e29-4410-83f1-ebbfc585b24e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:20:49.755658 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-3f0df024-9e29-4410-83f1-ebbfc585b24e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:20:49.768909 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0df024-9e29-4410-83f1-ebbfc585b24e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:20:49.771160 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0df024-9e29-4410-83f1-ebbfc585b24e service placement] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:20:49.782601 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1277/2535] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:20:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:20:49.910256 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed5e379-4866-4c3f-9e63-ce89fe2f6963 service nova] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:20:49.912603 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-1ed5e379-4866-4c3f-9e63-ce89fe2f6963 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:20:49.928257 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed5e379-4866-4c3f-9e63-ce89fe2f6963 service nova] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:20:49.933483 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33469c2-1075-47ae-9828-c1fa70680ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:20:49.934137 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33469c2-1075-47ae-9828-c1fa70680ccd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:20:49.934417 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33469c2-1075-47ae-9828-c1fa70680ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:20:49.934688 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33469c2-1075-47ae-9828-c1fa70680ccd admin admin] PATH_INFO: `/v3/users/ecdea0572cc64ca394cd4d977f536d5a` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:20:49.946248 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33469c2-1075-47ae-9828-c1fa70680ccd admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecdea0572cc64ca394cd4d977f536d5a)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:20:49.948823 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33469c2-1075-47ae-9828-c1fa70680ccd admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:20:50.143564 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-a33469c2-1075-47ae-9828-c1fa70680ccd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecdea0572cc64ca394cd4d977f536d5a'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:20:50.156826 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-a33469c2-1075-47ae-9828-c1fa70680ccd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecdea0572cc64ca394cd4d977f536d5a'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:20:50.163602 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-a33469c2-1075-47ae-9828-c1fa70680ccd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecdea0572cc64ca394cd4d977f536d5a'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:20:50.259462 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-a33469c2-1075-47ae-9828-c1fa70680ccd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecdea0572cc64ca394cd4d977f536d5a'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:20:50.260094 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-a33469c2-1075-47ae-9828-c1fa70680ccd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecdea0572cc64ca394cd4d977f536d5a'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:20:50.277654 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1259/2536] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:20:49 2022] DELETE /identity/v3/users/ecdea0572cc64ca394cd4d977f536d5a => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:20:50.537141 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0df024-9e29-4410-83f1-ebbfc585b24e service placement] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:20:50.555162 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0df024-9e29-4410-83f1-ebbfc585b24e service placement] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:20:50.615374 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8007255-99e8-4a89-ad5b-303f014bccca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:20:50.616725 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8007255-99e8-4a89-ad5b-303f014bccca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:20:50.617034 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8007255-99e8-4a89-ad5b-303f014bccca admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:20:50.617322 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8007255-99e8-4a89-ad5b-303f014bccca admin admin] PATH_INFO: `/v3/projects/289886000851431da2e3aebc3b47be8a` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:20:50.618808 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8007255-99e8-4a89-ad5b-303f014bccca admin admin] RBAC: Authorizing `identity:delete_project(project_id=289886000851431da2e3aebc3b47be8a)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:20:50.620839 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8007255-99e8-4a89-ad5b-303f014bccca admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:20:50.719978 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:20:50.729629 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-a8007255-99e8-4a89-ad5b-303f014bccca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '289886000851431da2e3aebc3b47be8a', 'request_id': 'req-a8007255-99e8-4a89-ad5b-303f014bccca'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:20:50.736539 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-a8007255-99e8-4a89-ad5b-303f014bccca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '289886000851431da2e3aebc3b47be8a', 'request_id': 'req-a8007255-99e8-4a89-ad5b-303f014bccca'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:20:50.837023 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-a8007255-99e8-4a89-ad5b-303f014bccca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '289886000851431da2e3aebc3b47be8a', 'request_id': 'req-a8007255-99e8-4a89-ad5b-303f014bccca'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:20:50.852832 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-a8007255-99e8-4a89-ad5b-303f014bccca admin admin] The token cache is being invalidate because project 289886000851431da2e3aebc3b47be8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:20:50.853124 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-a8007255-99e8-4a89-ad5b-303f014bccca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:20:50.854148 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-a8007255-99e8-4a89-ad5b-303f014bccca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:20:50.856148 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1278/2537] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:20:50 2022] DELETE /identity/v3/projects/289886000851431da2e3aebc3b47be8a => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:20:50.861020 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81974ffb-c942-4fe1-8585-8f951b52de0f None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:20:50.861304 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81974ffb-c942-4fe1-8585-8f951b52de0f None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:20:50.861644 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81974ffb-c942-4fe1-8585-8f951b52de0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:20:50.926716 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-81974ffb-c942-4fe1-8585-8f951b52de0f None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:20:50.929501 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-81974ffb-c942-4fe1-8585-8f951b52de0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:20:50.972250 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:20:51.005730 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1260/2538] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:20:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:20:51.015337 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fadd77d-f760-463c-8858-9ec6e83b5ba1 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:20:51.015700 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fadd77d-f760-463c-8858-9ec6e83b5ba1 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:20:51.016896 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fadd77d-f760-463c-8858-9ec6e83b5ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:20:51.060249 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-9fadd77d-f760-463c-8858-9ec6e83b5ba1 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:20:51.061727 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-9fadd77d-f760-463c-8858-9ec6e83b5ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:20:51.071174 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1279/2539] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:20:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:20:51.075737 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81974ffb-c942-4fe1-8585-8f951b52de0f None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:20:51.088652 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81974ffb-c942-4fe1-8585-8f951b52de0f None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:20:51.104421 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3affd7bc-34d3-4f5d-9991-d3f073d8d176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:20:51.105245 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3affd7bc-34d3-4f5d-9991-d3f073d8d176 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:20:51.105806 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3affd7bc-34d3-4f5d-9991-d3f073d8d176 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:20:51.106090 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3affd7bc-34d3-4f5d-9991-d3f073d8d176 admin admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:20:51.106755 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3affd7bc-34d3-4f5d-9991-d3f073d8d176 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:20:51.107154 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3affd7bc-34d3-4f5d-9991-d3f073d8d176 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:20:51.109000 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3affd7bc-34d3-4f5d-9991-d3f073d8d176 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:20:51.125804 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1261/2540] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:20:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 17:20:51.132537 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fadd77d-f760-463c-8858-9ec6e83b5ba1 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:20:51.155699 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fadd77d-f760-463c-8858-9ec6e83b5ba1 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:20:51.176110 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c821a840-4d3a-4736-a24c-829448809cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:20:51.180695 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c821a840-4d3a-4736-a24c-829448809cb4 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:20:51.181252 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c821a840-4d3a-4736-a24c-829448809cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:20:51.181816 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c821a840-4d3a-4736-a24c-829448809cb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:20:51.182583 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c821a840-4d3a-4736-a24c-829448809cb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:20:51.185103 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c821a840-4d3a-4736-a24c-829448809cb4 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:20:51.337423 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1280/2541] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:20:51 2022] POST /identity/v3/projects => generated 375 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:20:51.341443 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3affd7bc-34d3-4f5d-9991-d3f073d8d176 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:20:51.353129 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3affd7bc-34d3-4f5d-9991-d3f073d8d176 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:20:51.430882 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6879228-d0d9-442e-813d-71bc72556363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:20:51.431647 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6879228-d0d9-442e-813d-71bc72556363 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:20:51.432036 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6879228-d0d9-442e-813d-71bc72556363 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:20:51.432407 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6879228-d0d9-442e-813d-71bc72556363 admin admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:20:51.433506 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6879228-d0d9-442e-813d-71bc72556363 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:20:51.435304 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6879228-d0d9-442e-813d-71bc72556363 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:20:51.522708 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1262/2542] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:20:51 2022] POST /identity/v3/users => generated 288 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:20:51.527045 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c821a840-4d3a-4736-a24c-829448809cb4 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:20:51.537705 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c821a840-4d3a-4736-a24c-829448809cb4 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:20:51.542498 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e3e514-95b6-4910-9478-475ed3a8eb34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:20:51.543212 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e3e514-95b6-4910-9478-475ed3a8eb34 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:20:51.543494 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e3e514-95b6-4910-9478-475ed3a8eb34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:20:51.543766 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e3e514-95b6-4910-9478-475ed3a8eb34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:20:51.544411 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e3e514-95b6-4910-9478-475ed3a8eb34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:20:51.546057 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e3e514-95b6-4910-9478-475ed3a8eb34 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:20:51.557121 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1281/2543] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:20:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:20:51.561338 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6879228-d0d9-442e-813d-71bc72556363 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:20:51.572525 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6879228-d0d9-442e-813d-71bc72556363 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:20:51.577069 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06540e30-f5d0-4007-a855-a8cf15dd4954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:20:51.577668 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06540e30-f5d0-4007-a855-a8cf15dd4954 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:20:51.577949 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06540e30-f5d0-4007-a855-a8cf15dd4954 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:20:51.578224 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06540e30-f5d0-4007-a855-a8cf15dd4954 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:20:51.578854 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06540e30-f5d0-4007-a855-a8cf15dd4954 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:20:51.580487 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06540e30-f5d0-4007-a855-a8cf15dd4954 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:20:51.596338 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1263/2544] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:20:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:20:51.597684 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e3e514-95b6-4910-9478-475ed3a8eb34 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:20:51.609768 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e3e514-95b6-4910-9478-475ed3a8eb34 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:20:51.615327 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724e5aff-812f-4895-a3d1-a756d0d273e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:20:51.616103 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e5aff-812f-4895-a3d1-a756d0d273e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:20:51.616268 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e5aff-812f-4895-a3d1-a756d0d273e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:20:51.616448 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e5aff-812f-4895-a3d1-a756d0d273e4 admin admin] PATH_INFO: `/v3/projects/b531205569ec466f8e1ce8ce05427663/users/c5796bbaaf1345b6867d3cd5082f75b0/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:20:51.635233 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724e5aff-812f-4895-a3d1-a756d0d273e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b531205569ec466f8e1ce8ce05427663, user_id=c5796bbaaf1345b6867d3cd5082f75b0, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:20:51.641075 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724e5aff-812f-4895-a3d1-a756d0d273e4 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:20:51.713092 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1282/2545] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:20:51 2022] PUT /identity/v3/projects/b531205569ec466f8e1ce8ce05427663/users/c5796bbaaf1345b6867d3cd5082f75b0/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:20:51.718364 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5b00e4-8a54-4d90-aad7-fde4f8e35942 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:20:51.718685 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5b00e4-8a54-4d90-aad7-fde4f8e35942 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:20:51.718990 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5b00e4-8a54-4d90-aad7-fde4f8e35942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:20:51.749625 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-2d5b00e4-8a54-4d90-aad7-fde4f8e35942 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:20:51.750883 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-2d5b00e4-8a54-4d90-aad7-fde4f8e35942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:20:51.819615 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1264/2546] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:20:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:20:51.831015 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724e5aff-812f-4895-a3d1-a756d0d273e4 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:20:51.832890 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-724e5aff-812f-4895-a3d1-a756d0d273e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:20:51.894011 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724e5aff-812f-4895-a3d1-a756d0d273e4 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:20:51.899793 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb8739b-bb4c-4fff-85b5-0159aa7362c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:20:51.901071 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb8739b-bb4c-4fff-85b5-0159aa7362c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:20:51.901071 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb8739b-bb4c-4fff-85b5-0159aa7362c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:20:51.901071 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb8739b-bb4c-4fff-85b5-0159aa7362c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:20:51.910539 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb8739b-bb4c-4fff-85b5-0159aa7362c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:20:51.912040 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb8739b-bb4c-4fff-85b5-0159aa7362c7 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:20:51.921683 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1283/2547] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:20:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:20:52.441164 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e7f8ac-8d18-48d8-9580-1ea3bd630284 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:20:52.441665 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e7f8ac-8d18-48d8-9580-1ea3bd630284 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:20:52.441769 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e7f8ac-8d18-48d8-9580-1ea3bd630284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:20:52.461150 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-06e7f8ac-8d18-48d8-9580-1ea3bd630284 None None] MFA Rules not processed for user `04ab53c336934355a0000bda59fa7e7c`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:20:52.462394 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-06e7f8ac-8d18-48d8-9580-1ea3bd630284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:20:52.511814 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:20:52.533868 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1265/2548] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:20:52 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:20:53.782709 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9293245f-b6c2-42e6-b7e3-6b9a5b65863c None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:20:53.783582 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9293245f-b6c2-42e6-b7e3-6b9a5b65863c None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:20:53.790114 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9293245f-b6c2-42e6-b7e3-6b9a5b65863c None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:20:53.790114 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1284/2549] 10.222.0.82 () {54 vars in 921 bytes} [Thu Aug 18 17:20:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:20:53.792433 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18919a61-e3d0-4502-8386-67b555aaaf5d None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:20:53.792907 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18919a61-e3d0-4502-8386-67b555aaaf5d None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:20:53.793008 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18919a61-e3d0-4502-8386-67b555aaaf5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:20:53.842209 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-18919a61-e3d0-4502-8386-67b555aaaf5d None None] MFA Rules not processed for user `55723c756cae4e7ab636054db48de02d`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:20:53.843831 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-18919a61-e3d0-4502-8386-67b555aaaf5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:20:53.921828 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1266/2550] 10.222.0.82 () {58 vars in 1018 bytes} [Thu Aug 18 17:20:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:20:53.941184 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9293245f-b6c2-42e6-b7e3-6b9a5b65863c None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:20:53.956982 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9293245f-b6c2-42e6-b7e3-6b9a5b65863c None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:20:53.964621 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e766656f-125d-4b88-8795-4e5816f9ef4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:20:53.965403 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e766656f-125d-4b88-8795-4e5816f9ef4d service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:20:53.965928 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e766656f-125d-4b88-8795-4e5816f9ef4d service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:20:53.966432 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e766656f-125d-4b88-8795-4e5816f9ef4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:20:53.979021 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e766656f-125d-4b88-8795-4e5816f9ef4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:20:53.980758 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e766656f-125d-4b88-8795-4e5816f9ef4d service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:20:53.998367 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1285/2551] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:20:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:20:57.538803 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6fb0c7-776a-4c7f-bb89-a7a1e908e811 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:20:57.539387 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6fb0c7-776a-4c7f-bb89-a7a1e908e811 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:20:57.539387 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6fb0c7-776a-4c7f-bb89-a7a1e908e811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:20:57.575245 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-ab6fb0c7-776a-4c7f-bb89-a7a1e908e811 None None] MFA Rules not processed for user `c5796bbaaf1345b6867d3cd5082f75b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:20:57.577119 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-ab6fb0c7-776a-4c7f-bb89-a7a1e908e811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:20:57.657928 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1267/2552] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:20:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:20:57.673336 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e766656f-125d-4b88-8795-4e5816f9ef4d service neutron] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:20:57.674870 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-e766656f-125d-4b88-8795-4e5816f9ef4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:20:57.691369 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e766656f-125d-4b88-8795-4e5816f9ef4d service neutron] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:20:57.698248 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d823ef77-0557-4c81-8e71-cc9c59ad4018 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55723c756cae4e7ab636054db48de02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:20:57.698862 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823ef77-0557-4c81-8e71-cc9c59ad4018 service nova] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:20:57.699174 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823ef77-0557-4c81-8e71-cc9c59ad4018 service nova] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:20:57.699448 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823ef77-0557-4c81-8e71-cc9c59ad4018 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:20:57.708895 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d823ef77-0557-4c81-8e71-cc9c59ad4018 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:20:57.710623 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d823ef77-0557-4c81-8e71-cc9c59ad4018 service nova] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:20:57.720886 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1286/2553] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:20:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:21:00.426852 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a474d252-c1f1-4604-b3f7-017ff7248cad None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:21:00.429422 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a474d252-c1f1-4604-b3f7-017ff7248cad None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:21:00.429797 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a474d252-c1f1-4604-b3f7-017ff7248cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:21:00.449789 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-a474d252-c1f1-4604-b3f7-017ff7248cad None None] MFA Rules not processed for user `c5796bbaaf1345b6867d3cd5082f75b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:21:00.451051 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-a474d252-c1f1-4604-b3f7-017ff7248cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:21:00.479440 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1268/2554] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:21:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:21:07.716287 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:DELETE=5,keystone:SELECT=76,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:21:10.444508 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=79 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:22:11.537794 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5740c507-998e-4c4d-a71f-e83141087d6e None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:22:11.538833 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5740c507-998e-4c4d-a71f-e83141087d6e None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:22:11.538833 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5740c507-998e-4c4d-a71f-e83141087d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:22:11.557550 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-5740c507-998e-4c4d-a71f-e83141087d6e None None] MFA Rules not processed for user `5de19a54ed104bcdb35ee4e5e793f1e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:22:11.558736 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-5740c507-998e-4c4d-a71f-e83141087d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:22:11.615387 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1287/2555] 10.222.0.82 () {58 vars in 1009 bytes} [Thu Aug 18 17:22:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:22:11.749305 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a474d252-c1f1-4604-b3f7-017ff7248cad None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:22:11.750810 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-a474d252-c1f1-4604-b3f7-017ff7248cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:22:11.819023 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a474d252-c1f1-4604-b3f7-017ff7248cad None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:22:11.823962 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d04d322-9363-4b93-946e-66b1d420729c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1381673f81584f4b8084823e3aa99e2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:22:11.824568 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d04d322-9363-4b93-946e-66b1d420729c service swift] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:22:11.824854 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d04d322-9363-4b93-946e-66b1d420729c service swift] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:22:11.825128 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d04d322-9363-4b93-946e-66b1d420729c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:22:11.833645 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d04d322-9363-4b93-946e-66b1d420729c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:22:11.835277 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d04d322-9363-4b93-946e-66b1d420729c service swift] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:22:11.843151 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1269/2556] 10.222.0.82 () {60 vars in 1384 bytes} [Thu Aug 18 17:22:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 17:22:20.914571 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5740c507-998e-4c4d-a71f-e83141087d6e None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:22:20.916457 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-5740c507-998e-4c4d-a71f-e83141087d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:22:20.929057 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5740c507-998e-4c4d-a71f-e83141087d6e None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:22:20.932714 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac6d9b7-df44-421f-bef4-65e646f1a72d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:22:20.933329 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac6d9b7-df44-421f-bef4-65e646f1a72d service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:22:20.933623 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac6d9b7-df44-421f-bef4-65e646f1a72d service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:22:20.933877 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac6d9b7-df44-421f-bef4-65e646f1a72d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:22:20.935712 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-6ac6d9b7-df44-421f-bef4-65e646f1a72d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:22:20.950751 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac6d9b7-df44-421f-bef4-65e646f1a72d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:22:20.952196 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac6d9b7-df44-421f-bef4-65e646f1a72d service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:22:20.965240 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1288/2557] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:22:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:22:21.839997 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=32 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:22:22.173155 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d04d322-9363-4b93-946e-66b1d420729c service swift] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:22:22.185662 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d04d322-9363-4b93-946e-66b1d420729c service swift] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:22:22.190401 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d522de-bcb7-4726-a64c-4974ffc61735 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:22:22.191123 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d522de-bcb7-4726-a64c-4974ffc61735 service neutron] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:22:22.191411 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d522de-bcb7-4726-a64c-4974ffc61735 service neutron] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:22:22.191682 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d522de-bcb7-4726-a64c-4974ffc61735 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:22:22.193577 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-40d522de-bcb7-4726-a64c-4974ffc61735 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:22:22.209256 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d522de-bcb7-4726-a64c-4974ffc61735 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:22:22.210848 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d522de-bcb7-4726-a64c-4974ffc61735 service neutron] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:22:22.225215 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1270/2558] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:22:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:22:29.962292 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac6d9b7-df44-421f-bef4-65e646f1a72d service neutron] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:22:29.974304 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac6d9b7-df44-421f-bef4-65e646f1a72d service neutron] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:22:29.979813 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c1b489-091a-4717-819c-4af4be51e11a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55723c756cae4e7ab636054db48de02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:22:29.980405 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c1b489-091a-4717-819c-4af4be51e11a service nova] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:22:29.980713 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c1b489-091a-4717-819c-4af4be51e11a service nova] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:22:29.980989 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c1b489-091a-4717-819c-4af4be51e11a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:22:29.982679 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-a1c1b489-091a-4717-819c-4af4be51e11a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:22:30.062985 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c1b489-091a-4717-819c-4af4be51e11a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:22:30.064552 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c1b489-091a-4717-819c-4af4be51e11a service nova] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:22:30.073647 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1289/2559] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:22:29 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:22:30.530173 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2a756-6f0e-4df3-8971-5e9fc5db53f0 None None] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:22:30.530757 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2a756-6f0e-4df3-8971-5e9fc5db53f0 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:22:30.530968 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2a756-6f0e-4df3-8971-5e9fc5db53f0 None None] PATH_INFO: `/` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:22:30.532347 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1271/2560] 10.222.0.82 () {54 vars in 921 bytes} [Thu Aug 18 17:22:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:22:30.538996 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7abdafb-a9c0-40ac-84d2-4996c461935d None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:22:30.539352 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7abdafb-a9c0-40ac-84d2-4996c461935d None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:22:30.539702 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7abdafb-a9c0-40ac-84d2-4996c461935d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:22:30.578277 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-f7abdafb-a9c0-40ac-84d2-4996c461935d None None] MFA Rules not processed for user `55723c756cae4e7ab636054db48de02d`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:22:30.579514 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-f7abdafb-a9c0-40ac-84d2-4996c461935d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:22:30.589161 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1290/2561] 10.222.0.82 () {58 vars in 1018 bytes} [Thu Aug 18 17:22:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:22:30.601178 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d2a756-6f0e-4df3-8971-5e9fc5db53f0 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:22:30.602930 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-39d2a756-6f0e-4df3-8971-5e9fc5db53f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:22:30.659706 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d2a756-6f0e-4df3-8971-5e9fc5db53f0 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:22:30.663146 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e0ba56-289d-440d-83a5-e27c7612a119 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bee1ba6bb324de4915904f83349b3c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:22:30.663763 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e0ba56-289d-440d-83a5-e27c7612a119 service glance] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:22:30.664042 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e0ba56-289d-440d-83a5-e27c7612a119 service glance] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:22:30.664336 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e0ba56-289d-440d-83a5-e27c7612a119 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:22:30.674118 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e0ba56-289d-440d-83a5-e27c7612a119 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:22:30.675593 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e0ba56-289d-440d-83a5-e27c7612a119 service glance] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:22:30.686522 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1272/2562] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:22:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 17:22:30.861554 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d61ac-7781-4186-8c2b-0fb6ca71f167 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:22:30.861918 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d61ac-7781-4186-8c2b-0fb6ca71f167 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:22:30.862197 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d61ac-7781-4186-8c2b-0fb6ca71f167 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:22:30.863589 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1291/2563] 10.222.0.82 () {54 vars in 921 bytes} [Thu Aug 18 17:22:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 17:22:30.868657 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf898c-5908-4c4a-bae1-fec446b2233f None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:22:30.868941 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf898c-5908-4c4a-bae1-fec446b2233f None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:22:30.869215 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf898c-5908-4c4a-bae1-fec446b2233f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:22:30.888743 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-84cf898c-5908-4c4a-bae1-fec446b2233f None None] MFA Rules not processed for user `fe2403ede53148b2a113ffa2a605e987`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:22:30.889927 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-84cf898c-5908-4c4a-bae1-fec446b2233f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:22:30.897049 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1273/2564] 10.222.0.82 () {58 vars in 1018 bytes} [Thu Aug 18 17:22:30 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 17:22:30.909037 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2d61ac-7781-4186-8c2b-0fb6ca71f167 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:22:30.919671 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2d61ac-7781-4186-8c2b-0fb6ca71f167 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:22:30.923053 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12884472-243e-4fbb-b604-4e334bde031f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:22:30.923631 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12884472-243e-4fbb-b604-4e334bde031f service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:22:30.923917 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12884472-243e-4fbb-b604-4e334bde031f service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:22:30.924185 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12884472-243e-4fbb-b604-4e334bde031f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:22:30.930942 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12884472-243e-4fbb-b604-4e334bde031f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:22:30.932348 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12884472-243e-4fbb-b604-4e334bde031f service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:22:30.939806 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1292/2565] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:22:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:22:33.191816 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcfacdd-ec5e-4cfb-8c22-5e5c3edf30b6 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:22:33.192277 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcfacdd-ec5e-4cfb-8c22-5e5c3edf30b6 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:22:33.192384 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcfacdd-ec5e-4cfb-8c22-5e5c3edf30b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:22:33.211349 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-cbcfacdd-ec5e-4cfb-8c22-5e5c3edf30b6 None None] MFA Rules not processed for user `5de19a54ed104bcdb35ee4e5e793f1e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:22:33.212562 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-cbcfacdd-ec5e-4cfb-8c22-5e5c3edf30b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:22:33.220998 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1274/2566] 10.222.0.82 () {58 vars in 1009 bytes} [Thu Aug 18 17:22:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:22:33.233459 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12884472-243e-4fbb-b604-4e334bde031f service neutron] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:22:33.245851 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12884472-243e-4fbb-b604-4e334bde031f service neutron] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:22:33.250652 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532d85f3-4e17-48ac-a2aa-d5f65da45289 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1381673f81584f4b8084823e3aa99e2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:22:33.251271 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d85f3-4e17-48ac-a2aa-d5f65da45289 service swift] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:22:33.251548 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d85f3-4e17-48ac-a2aa-d5f65da45289 service swift] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:22:33.251842 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d85f3-4e17-48ac-a2aa-d5f65da45289 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:22:33.259826 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532d85f3-4e17-48ac-a2aa-d5f65da45289 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:22:33.261369 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532d85f3-4e17-48ac-a2aa-d5f65da45289 service swift] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:22:33.269731 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1293/2567] 10.222.0.82 () {60 vars in 1384 bytes} [Thu Aug 18 17:22:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 17:22:43.205641 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=38 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:22:43.265953 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=89 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:22:52.080240 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcfacdd-ec5e-4cfb-8c22-5e5c3edf30b6 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:22:52.089728 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcfacdd-ec5e-4cfb-8c22-5e5c3edf30b6 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:22:52.093423 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72b8abc-3c3c-43c9-a572-9ad7497b602b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:22:52.094056 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b8abc-3c3c-43c9-a572-9ad7497b602b service neutron] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:22:52.094319 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b8abc-3c3c-43c9-a572-9ad7497b602b service neutron] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:22:52.094594 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b8abc-3c3c-43c9-a572-9ad7497b602b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:22:52.096354 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-d72b8abc-3c3c-43c9-a572-9ad7497b602b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:22:52.106325 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72b8abc-3c3c-43c9-a572-9ad7497b602b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:22:52.107846 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72b8abc-3c3c-43c9-a572-9ad7497b602b service neutron] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:22:52.116699 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1275/2568] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:22:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:22:53.981326 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6439ca-35ab-4079-bd3e-88687cf39572 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:22:53.981804 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6439ca-35ab-4079-bd3e-88687cf39572 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:22:53.981907 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6439ca-35ab-4079-bd3e-88687cf39572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:22:54.032014 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-ee6439ca-35ab-4079-bd3e-88687cf39572 None None] MFA Rules not processed for user `5de19a54ed104bcdb35ee4e5e793f1e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:22:54.033224 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-ee6439ca-35ab-4079-bd3e-88687cf39572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:22:54.050116 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1294/2569] 10.222.0.82 () {58 vars in 967 bytes} [Thu Aug 18 17:22:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:22:54.062696 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72b8abc-3c3c-43c9-a572-9ad7497b602b service neutron] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:22:54.085330 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72b8abc-3c3c-43c9-a572-9ad7497b602b service neutron] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:22:54.090542 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937d2f02-cae3-45ca-b5d9-9a5c0cef4483 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1381673f81584f4b8084823e3aa99e2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:22:54.091181 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d2f02-cae3-45ca-b5d9-9a5c0cef4483 service swift] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:22:54.091467 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d2f02-cae3-45ca-b5d9-9a5c0cef4483 service swift] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:22:54.091742 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d2f02-cae3-45ca-b5d9-9a5c0cef4483 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:22:54.100325 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937d2f02-cae3-45ca-b5d9-9a5c0cef4483 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:22:54.101854 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937d2f02-cae3-45ca-b5d9-9a5c0cef4483 service swift] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:22:54.110260 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1276/2570] 10.222.0.82 () {60 vars in 1384 bytes} [Thu Aug 18 17:22:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 17:23:01.561360 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6439ca-35ab-4079-bd3e-88687cf39572 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:23:01.570888 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6439ca-35ab-4079-bd3e-88687cf39572 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:23:01.574423 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7930dc1-0623-40ae-84f6-8c4f73f7842f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:23:01.575063 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7930dc1-0623-40ae-84f6-8c4f73f7842f service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:23:01.575345 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7930dc1-0623-40ae-84f6-8c4f73f7842f service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:23:01.575617 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7930dc1-0623-40ae-84f6-8c4f73f7842f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:23:01.577318 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-a7930dc1-0623-40ae-84f6-8c4f73f7842f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:23:01.588718 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7930dc1-0623-40ae-84f6-8c4f73f7842f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:23:01.590122 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7930dc1-0623-40ae-84f6-8c4f73f7842f service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:23:01.599918 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1295/2571] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:23:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:23:04.106749 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=16 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:23:09.010721 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937d2f02-cae3-45ca-b5d9-9a5c0cef4483 service swift] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:23:09.012274 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-937d2f02-cae3-45ca-b5d9-9a5c0cef4483 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:23:09.032192 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937d2f02-cae3-45ca-b5d9-9a5c0cef4483 service swift] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:23:09.037232 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70de3e48-42ba-4c7a-98d5-596840c1899a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:23:09.037908 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70de3e48-42ba-4c7a-98d5-596840c1899a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:23:09.038196 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70de3e48-42ba-4c7a-98d5-596840c1899a admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:23:09.038475 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70de3e48-42ba-4c7a-98d5-596840c1899a admin admin] PATH_INFO: `/v3/users/ebc5df40911b4186abec99c01e0702a6` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:23:09.039622 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70de3e48-42ba-4c7a-98d5-596840c1899a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebc5df40911b4186abec99c01e0702a6)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:23:09.041638 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70de3e48-42ba-4c7a-98d5-596840c1899a admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:23:09.238968 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-70de3e48-42ba-4c7a-98d5-596840c1899a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebc5df40911b4186abec99c01e0702a6'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:23:09.251421 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-70de3e48-42ba-4c7a-98d5-596840c1899a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebc5df40911b4186abec99c01e0702a6'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:23:09.252088 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-70de3e48-42ba-4c7a-98d5-596840c1899a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebc5df40911b4186abec99c01e0702a6'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:23:09.316700 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-70de3e48-42ba-4c7a-98d5-596840c1899a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebc5df40911b4186abec99c01e0702a6'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:23:09.317319 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-70de3e48-42ba-4c7a-98d5-596840c1899a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebc5df40911b4186abec99c01e0702a6'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:23:09.330516 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1277/2572] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:23:09 2022] DELETE /identity/v3/users/ebc5df40911b4186abec99c01e0702a6 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:23:09.616850 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7930dc1-0623-40ae-84f6-8c4f73f7842f service neutron] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:23:09.627500 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7930dc1-0623-40ae-84f6-8c4f73f7842f service neutron] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:23:09.685150 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c86d706-7782-4887-8fda-d6437a04aaa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:23:09.685776 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86d706-7782-4887-8fda-d6437a04aaa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:23:09.686054 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86d706-7782-4887-8fda-d6437a04aaa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:23:09.686323 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86d706-7782-4887-8fda-d6437a04aaa2 admin admin] PATH_INFO: `/v3/projects/f8802bed1f6b4be991ff011f40bf8622` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:23:09.687503 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c86d706-7782-4887-8fda-d6437a04aaa2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8802bed1f6b4be991ff011f40bf8622)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:23:09.689382 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c86d706-7782-4887-8fda-d6437a04aaa2 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:23:09.793613 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-1c86d706-7782-4887-8fda-d6437a04aaa2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8802bed1f6b4be991ff011f40bf8622', 'request_id': 'req-1c86d706-7782-4887-8fda-d6437a04aaa2'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:23:09.799796 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-1c86d706-7782-4887-8fda-d6437a04aaa2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8802bed1f6b4be991ff011f40bf8622', 'request_id': 'req-1c86d706-7782-4887-8fda-d6437a04aaa2'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:23:09.894203 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-1c86d706-7782-4887-8fda-d6437a04aaa2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8802bed1f6b4be991ff011f40bf8622', 'request_id': 'req-1c86d706-7782-4887-8fda-d6437a04aaa2'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:23:09.904993 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-1c86d706-7782-4887-8fda-d6437a04aaa2 admin admin] The token cache is being invalidate because project f8802bed1f6b4be991ff011f40bf8622 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:23:09.905279 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-1c86d706-7782-4887-8fda-d6437a04aaa2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:23:09.905942 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-1c86d706-7782-4887-8fda-d6437a04aaa2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:23:09.907533 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1296/2573] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:23:09 2022] DELETE /identity/v3/projects/f8802bed1f6b4be991ff011f40bf8622 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:23:09.912144 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70de3e48-42ba-4c7a-98d5-596840c1899a admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:23:09.913783 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-70de3e48-42ba-4c7a-98d5-596840c1899a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:23:09.926581 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70de3e48-42ba-4c7a-98d5-596840c1899a admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:23:09.983044 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ff7b6b-1b02-4843-9a59-671da7542f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:23:09.983670 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ff7b6b-1b02-4843-9a59-671da7542f24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:23:09.983951 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ff7b6b-1b02-4843-9a59-671da7542f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:23:09.984224 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ff7b6b-1b02-4843-9a59-671da7542f24 admin admin] PATH_INFO: `/v3/users/6af30f5ad30b433983e90592fd3f4d8c` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:23:09.985318 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ff7b6b-1b02-4843-9a59-671da7542f24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6af30f5ad30b433983e90592fd3f4d8c)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:23:09.987263 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ff7b6b-1b02-4843-9a59-671da7542f24 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:23:10.150610 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-95ff7b6b-1b02-4843-9a59-671da7542f24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6af30f5ad30b433983e90592fd3f4d8c'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:23:10.164180 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-95ff7b6b-1b02-4843-9a59-671da7542f24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6af30f5ad30b433983e90592fd3f4d8c'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:23:10.164876 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-95ff7b6b-1b02-4843-9a59-671da7542f24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6af30f5ad30b433983e90592fd3f4d8c'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:23:10.231705 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-95ff7b6b-1b02-4843-9a59-671da7542f24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6af30f5ad30b433983e90592fd3f4d8c'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:23:10.232471 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-95ff7b6b-1b02-4843-9a59-671da7542f24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6af30f5ad30b433983e90592fd3f4d8c'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:23:10.234791 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:23:10.246226 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1278/2574] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:23:09 2022] DELETE /identity/v3/users/6af30f5ad30b433983e90592fd3f4d8c => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:23:10.524977 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c86d706-7782-4887-8fda-d6437a04aaa2 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:23:10.526455 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-1c86d706-7782-4887-8fda-d6437a04aaa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:23:10.541446 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c86d706-7782-4887-8fda-d6437a04aaa2 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:23:10.576981 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:23:10.599323 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cfab63-f198-477e-9822-08f767a25145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:23:10.599891 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfab63-f198-477e-9822-08f767a25145 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:23:10.600167 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfab63-f198-477e-9822-08f767a25145 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:23:10.601059 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfab63-f198-477e-9822-08f767a25145 admin admin] PATH_INFO: `/v3/projects/5db98e82fc6f4234a6b7bc57139ec073` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:23:10.602478 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cfab63-f198-477e-9822-08f767a25145 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5db98e82fc6f4234a6b7bc57139ec073)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:23:10.604479 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cfab63-f198-477e-9822-08f767a25145 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:23:10.695724 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-36cfab63-f198-477e-9822-08f767a25145 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5db98e82fc6f4234a6b7bc57139ec073', 'request_id': 'req-36cfab63-f198-477e-9822-08f767a25145'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:23:10.702864 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-36cfab63-f198-477e-9822-08f767a25145 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5db98e82fc6f4234a6b7bc57139ec073', 'request_id': 'req-36cfab63-f198-477e-9822-08f767a25145'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:23:10.792134 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-36cfab63-f198-477e-9822-08f767a25145 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5db98e82fc6f4234a6b7bc57139ec073', 'request_id': 'req-36cfab63-f198-477e-9822-08f767a25145'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:23:10.803230 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-36cfab63-f198-477e-9822-08f767a25145 admin admin] The token cache is being invalidate because project 5db98e82fc6f4234a6b7bc57139ec073 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:23:10.803721 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-36cfab63-f198-477e-9822-08f767a25145 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:23:10.804420 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-36cfab63-f198-477e-9822-08f767a25145 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:23:10.805929 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1297/2575] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:23:10 2022] DELETE /identity/v3/projects/5db98e82fc6f4234a6b7bc57139ec073 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:23:20.391688 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=4 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:23:20.872598 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=34,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:23:34.017489 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d54ad9-d197-47a6-a93a-af3bf8dcbfa8 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:23:34.018601 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d54ad9-d197-47a6-a93a-af3bf8dcbfa8 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:23:34.018601 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d54ad9-d197-47a6-a93a-af3bf8dcbfa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:23:34.036789 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-f2d54ad9-d197-47a6-a93a-af3bf8dcbfa8 None None] MFA Rules not processed for user `04ab53c336934355a0000bda59fa7e7c`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:23:34.038022 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-f2d54ad9-d197-47a6-a93a-af3bf8dcbfa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:23:34.092343 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1279/2576] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:23:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:23:35.092206 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cfab63-f198-477e-9822-08f767a25145 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:23:35.093650 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-36cfab63-f198-477e-9822-08f767a25145 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:23:35.172610 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cfab63-f198-477e-9822-08f767a25145 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:23:35.177454 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafeb535-4765-4e84-b7ba-dfcfec1f8328 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29653ba1a2a74232972ef7a9000518b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:23:35.178053 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafeb535-4765-4e84-b7ba-dfcfec1f8328 service placement] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:23:35.178330 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafeb535-4765-4e84-b7ba-dfcfec1f8328 service placement] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:23:35.178601 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafeb535-4765-4e84-b7ba-dfcfec1f8328 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:23:35.180402 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-eafeb535-4765-4e84-b7ba-dfcfec1f8328 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:23:35.191790 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafeb535-4765-4e84-b7ba-dfcfec1f8328 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:23:35.193343 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafeb535-4765-4e84-b7ba-dfcfec1f8328 service placement] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:23:35.203939 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1298/2577] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:23:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:23:44.086104 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=25 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:23:45.199553 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=32 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:23:52.072858 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d54ad9-d197-47a6-a93a-af3bf8dcbfa8 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:23:52.074558 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-f2d54ad9-d197-47a6-a93a-af3bf8dcbfa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:23:52.151423 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d54ad9-d197-47a6-a93a-af3bf8dcbfa8 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:23:52.156975 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9637660-d7ac-45dc-95e4-c9f03e007657 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55723c756cae4e7ab636054db48de02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:23:52.157588 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9637660-d7ac-45dc-95e4-c9f03e007657 service nova] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:23:52.157872 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9637660-d7ac-45dc-95e4-c9f03e007657 service nova] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:23:52.158145 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9637660-d7ac-45dc-95e4-c9f03e007657 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:23:52.159905 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-b9637660-d7ac-45dc-95e4-c9f03e007657 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:23:52.173042 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9637660-d7ac-45dc-95e4-c9f03e007657 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:23:52.174660 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9637660-d7ac-45dc-95e4-c9f03e007657 service nova] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:23:52.185103 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1280/2578] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:23:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:23:55.407375 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafeb535-4765-4e84-b7ba-dfcfec1f8328 service placement] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:23:55.408843 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-eafeb535-4765-4e84-b7ba-dfcfec1f8328 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:23:55.464946 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafeb535-4765-4e84-b7ba-dfcfec1f8328 service placement] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:23:55.468602 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf07faf-fa27-45b8-acf8-18b24adb39d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:23:55.469202 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf07faf-fa27-45b8-acf8-18b24adb39d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:23:55.469484 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf07faf-fa27-45b8-acf8-18b24adb39d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:23:55.469762 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf07faf-fa27-45b8-acf8-18b24adb39d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:23:55.471480 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-dbf07faf-fa27-45b8-acf8-18b24adb39d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:23:55.533664 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf07faf-fa27-45b8-acf8-18b24adb39d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:23:55.535117 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf07faf-fa27-45b8-acf8-18b24adb39d4 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:23:55.545201 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1299/2579] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:23:55 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:24:02.180708 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=34 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:24:05.540933 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=50 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:24:20.700879 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9637660-d7ac-45dc-95e4-c9f03e007657 service nova] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:24:20.710379 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9637660-d7ac-45dc-95e4-c9f03e007657 service nova] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:24:20.714961 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f872ba1-3a6e-4392-aa81-2f7d82694239 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:24:20.715558 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f872ba1-3a6e-4392-aa81-2f7d82694239 service neutron] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:24:20.715841 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f872ba1-3a6e-4392-aa81-2f7d82694239 service neutron] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:24:20.716118 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f872ba1-3a6e-4392-aa81-2f7d82694239 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:24:20.717830 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-7f872ba1-3a6e-4392-aa81-2f7d82694239 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:24:20.781288 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f872ba1-3a6e-4392-aa81-2f7d82694239 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:24:20.782743 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f872ba1-3a6e-4392-aa81-2f7d82694239 service neutron] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:24:20.793226 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1281/2580] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:24:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:24:24.305315 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf07faf-fa27-45b8-acf8-18b24adb39d4 service neutron] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:24:24.307498 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-dbf07faf-fa27-45b8-acf8-18b24adb39d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:24:24.323248 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf07faf-fa27-45b8-acf8-18b24adb39d4 service neutron] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:24:24.328040 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f440a451-3047-4c4a-bb6a-bdb163eccff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:24:24.328706 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f440a451-3047-4c4a-bb6a-bdb163eccff2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:24:24.328984 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f440a451-3047-4c4a-bb6a-bdb163eccff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:24:24.329251 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f440a451-3047-4c4a-bb6a-bdb163eccff2 admin admin] PATH_INFO: `/v3/users/22e737cba90b4d8890ecd692e6456470` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:24:24.330384 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f440a451-3047-4c4a-bb6a-bdb163eccff2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22e737cba90b4d8890ecd692e6456470)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:24:24.332341 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f440a451-3047-4c4a-bb6a-bdb163eccff2 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:24:24.496070 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-f440a451-3047-4c4a-bb6a-bdb163eccff2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22e737cba90b4d8890ecd692e6456470'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:24:24.568206 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-f440a451-3047-4c4a-bb6a-bdb163eccff2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22e737cba90b4d8890ecd692e6456470'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:24:24.568823 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-f440a451-3047-4c4a-bb6a-bdb163eccff2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22e737cba90b4d8890ecd692e6456470'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:24:24.569426 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-f440a451-3047-4c4a-bb6a-bdb163eccff2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22e737cba90b4d8890ecd692e6456470'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:24:24.576946 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-f440a451-3047-4c4a-bb6a-bdb163eccff2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22e737cba90b4d8890ecd692e6456470'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:24:24.594270 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1300/2581] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:24:24 2022] DELETE /identity/v3/users/22e737cba90b4d8890ecd692e6456470 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:24:24.816250 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f872ba1-3a6e-4392-aa81-2f7d82694239 service neutron] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:24:24.827737 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f872ba1-3a6e-4392-aa81-2f7d82694239 service neutron] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:24:24.883999 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efb93c5-3959-4dcb-924e-098a07b07071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:24:24.884626 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efb93c5-3959-4dcb-924e-098a07b07071 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:24:24.885517 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efb93c5-3959-4dcb-924e-098a07b07071 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:24:24.885974 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efb93c5-3959-4dcb-924e-098a07b07071 admin admin] PATH_INFO: `/v3/projects/d1d1a4ac5307416f8569267bf2f2d827` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:24:24.887222 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efb93c5-3959-4dcb-924e-098a07b07071 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1d1a4ac5307416f8569267bf2f2d827)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:24:24.889176 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efb93c5-3959-4dcb-924e-098a07b07071 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:24:24.996604 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-0efb93c5-3959-4dcb-924e-098a07b07071 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1d1a4ac5307416f8569267bf2f2d827', 'request_id': 'req-0efb93c5-3959-4dcb-924e-098a07b07071'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:24:25.070773 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-0efb93c5-3959-4dcb-924e-098a07b07071 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1d1a4ac5307416f8569267bf2f2d827', 'request_id': 'req-0efb93c5-3959-4dcb-924e-098a07b07071'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:24:25.077318 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-0efb93c5-3959-4dcb-924e-098a07b07071 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1d1a4ac5307416f8569267bf2f2d827', 'request_id': 'req-0efb93c5-3959-4dcb-924e-098a07b07071'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:24:25.088809 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-0efb93c5-3959-4dcb-924e-098a07b07071 admin admin] The token cache is being invalidate because project d1d1a4ac5307416f8569267bf2f2d827 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81650) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:24:25.089252 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-0efb93c5-3959-4dcb-924e-098a07b07071 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:24:25.089888 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-0efb93c5-3959-4dcb-924e-098a07b07071 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:24:25.091392 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1282/2582] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:24:24 2022] DELETE /identity/v3/projects/d1d1a4ac5307416f8569267bf2f2d827 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:24:25.096310 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c25cb-02f9-4435-abfe-012c558be6b5 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:24:25.096686 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c25cb-02f9-4435-abfe-012c558be6b5 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:24:25.097008 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c25cb-02f9-4435-abfe-012c558be6b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:24:25.117679 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-7a8c25cb-02f9-4435-abfe-012c558be6b5 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:24:25.118865 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-7a8c25cb-02f9-4435-abfe-012c558be6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:24:25.176647 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1301/2583] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:24:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:24:25.185172 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696338ef-ba1a-4897-b256-f1bbb3b8375e None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:24:25.185461 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696338ef-ba1a-4897-b256-f1bbb3b8375e None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:24:25.185743 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696338ef-ba1a-4897-b256-f1bbb3b8375e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:24:25.205893 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-696338ef-ba1a-4897-b256-f1bbb3b8375e None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:24:25.207118 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-696338ef-ba1a-4897-b256-f1bbb3b8375e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:24:25.215188 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1283/2584] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:24:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:24:25.219247 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8c25cb-02f9-4435-abfe-012c558be6b5 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:24:25.228947 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8c25cb-02f9-4435-abfe-012c558be6b5 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:24:25.233540 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b88887e-a0a5-4999-ac81-af4dc8d8fa6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:24:25.234137 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b88887e-a0a5-4999-ac81-af4dc8d8fa6b admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:24:25.234421 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b88887e-a0a5-4999-ac81-af4dc8d8fa6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:24:25.234695 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b88887e-a0a5-4999-ac81-af4dc8d8fa6b admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:24:25.235375 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b88887e-a0a5-4999-ac81-af4dc8d8fa6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:24:25.235679 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b88887e-a0a5-4999-ac81-af4dc8d8fa6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:24:25.237263 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b88887e-a0a5-4999-ac81-af4dc8d8fa6b admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:24:25.249664 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1302/2585] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:24:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 17:24:25.253946 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696338ef-ba1a-4897-b256-f1bbb3b8375e None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:24:25.263637 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696338ef-ba1a-4897-b256-f1bbb3b8375e None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:24:25.268297 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcff9ae-5f2d-4d51-b80f-68c4df9ff0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:24:25.268889 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcff9ae-5f2d-4d51-b80f-68c4df9ff0d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:24:25.269171 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcff9ae-5f2d-4d51-b80f-68c4df9ff0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:24:25.269448 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcff9ae-5f2d-4d51-b80f-68c4df9ff0d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:24:25.270070 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcff9ae-5f2d-4d51-b80f-68c4df9ff0d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:24:25.272027 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcff9ae-5f2d-4d51-b80f-68c4df9ff0d9 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:24:25.287025 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:24:25.394069 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1284/2586] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:24:25 2022] POST /identity/v3/projects => generated 385 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:24:25.402061 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b88887e-a0a5-4999-ac81-af4dc8d8fa6b admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:24:25.412084 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b88887e-a0a5-4999-ac81-af4dc8d8fa6b admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:24:25.468888 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20c5382-2ac3-4e35-aa30-cde01c245371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:24:25.469488 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20c5382-2ac3-4e35-aa30-cde01c245371 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:24:25.469859 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20c5382-2ac3-4e35-aa30-cde01c245371 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:24:25.470131 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20c5382-2ac3-4e35-aa30-cde01c245371 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:24:25.470739 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20c5382-2ac3-4e35-aa30-cde01c245371 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:24:25.472658 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20c5382-2ac3-4e35-aa30-cde01c245371 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:24:25.480548 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:24:25.568602 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1303/2587] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:24:25 2022] POST /identity/v3/users => generated 293 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:24:25.573288 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcff9ae-5f2d-4d51-b80f-68c4df9ff0d9 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:24:25.583520 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcff9ae-5f2d-4d51-b80f-68c4df9ff0d9 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:24:25.588117 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ccc99-de9a-4dc0-b2b0-4be3fcc13877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:24:25.588714 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ccc99-de9a-4dc0-b2b0-4be3fcc13877 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:24:25.588990 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ccc99-de9a-4dc0-b2b0-4be3fcc13877 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:24:25.589261 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ccc99-de9a-4dc0-b2b0-4be3fcc13877 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:24:25.589956 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4ccc99-de9a-4dc0-b2b0-4be3fcc13877 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:24:25.591729 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4ccc99-de9a-4dc0-b2b0-4be3fcc13877 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:24:25.601569 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1285/2588] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:24:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:24:25.606426 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20c5382-2ac3-4e35-aa30-cde01c245371 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:24:25.616392 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20c5382-2ac3-4e35-aa30-cde01c245371 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:24:25.622077 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9f8ebe-9356-4fc4-bc45-d4db98c47253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:24:25.622681 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f8ebe-9356-4fc4-bc45-d4db98c47253 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:24:25.622984 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f8ebe-9356-4fc4-bc45-d4db98c47253 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:24:25.623260 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f8ebe-9356-4fc4-bc45-d4db98c47253 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:24:25.623888 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9f8ebe-9356-4fc4-bc45-d4db98c47253 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:24:25.625476 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9f8ebe-9356-4fc4-bc45-d4db98c47253 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:24:25.635852 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1304/2589] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:24:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:24:25.639895 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ccc99-de9a-4dc0-b2b0-4be3fcc13877 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:24:25.650111 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ccc99-de9a-4dc0-b2b0-4be3fcc13877 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:24:25.654866 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ab5ecb-3e9e-4dd4-96b4-6f2aeac658f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:24:25.655829 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab5ecb-3e9e-4dd4-96b4-6f2aeac658f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:24:25.656119 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab5ecb-3e9e-4dd4-96b4-6f2aeac658f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:24:25.656398 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab5ecb-3e9e-4dd4-96b4-6f2aeac658f5 admin admin] PATH_INFO: `/v3/projects/808690fdba2741df889e0a21027606f9/users/da44ae3a4e8b41d3b8a3193db939e82d/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:24:25.671920 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ab5ecb-3e9e-4dd4-96b4-6f2aeac658f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=808690fdba2741df889e0a21027606f9, user_id=da44ae3a4e8b41d3b8a3193db939e82d, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:24:25.674609 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ab5ecb-3e9e-4dd4-96b4-6f2aeac658f5 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:24:25.739207 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1286/2590] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:24:25 2022] PUT /identity/v3/projects/808690fdba2741df889e0a21027606f9/users/da44ae3a4e8b41d3b8a3193db939e82d/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:24:25.744942 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f7e37-00eb-4f39-8dd0-d3af77755363 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:24:25.745233 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f7e37-00eb-4f39-8dd0-d3af77755363 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:24:25.745508 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f7e37-00eb-4f39-8dd0-d3af77755363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:24:25.763991 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-861f7e37-00eb-4f39-8dd0-d3af77755363 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:24:25.765908 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-861f7e37-00eb-4f39-8dd0-d3af77755363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:24:25.824083 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1305/2591] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:24:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:24:25.834600 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ab5ecb-3e9e-4dd4-96b4-6f2aeac658f5 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:24:25.836305 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-08ab5ecb-3e9e-4dd4-96b4-6f2aeac658f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:24:25.893735 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ab5ecb-3e9e-4dd4-96b4-6f2aeac658f5 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:24:25.897320 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe9880f-4c71-4c2a-ba5c-2e512274009e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:24:25.897920 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe9880f-4c71-4c2a-ba5c-2e512274009e service neutron] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:24:25.898197 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe9880f-4c71-4c2a-ba5c-2e512274009e service neutron] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:24:25.898468 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe9880f-4c71-4c2a-ba5c-2e512274009e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:24:25.907924 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe9880f-4c71-4c2a-ba5c-2e512274009e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:24:25.909293 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe9880f-4c71-4c2a-ba5c-2e512274009e service neutron] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:24:25.918570 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1287/2592] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:24:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:24:31.136223 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea12458-861d-447d-b153-478bef49dc60 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:24:31.136728 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea12458-861d-447d-b153-478bef49dc60 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:24:31.136841 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea12458-861d-447d-b153-478bef49dc60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:24:31.168218 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-aea12458-861d-447d-b153-478bef49dc60 None None] MFA Rules not processed for user `da44ae3a4e8b41d3b8a3193db939e82d`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:24:31.169432 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-aea12458-861d-447d-b153-478bef49dc60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:24:31.237010 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1306/2593] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:24:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:24:31.252816 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe9880f-4c71-4c2a-ba5c-2e512274009e service neutron] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:24:31.254285 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-afe9880f-4c71-4c2a-ba5c-2e512274009e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:24:31.332583 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe9880f-4c71-4c2a-ba5c-2e512274009e service neutron] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:24:31.338710 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee0e7bd-a92d-4b02-8de8-a6d40d7fa178 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55723c756cae4e7ab636054db48de02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:24:31.339590 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0e7bd-a92d-4b02-8de8-a6d40d7fa178 service nova] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:24:31.339867 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0e7bd-a92d-4b02-8de8-a6d40d7fa178 service nova] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:24:31.342487 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0e7bd-a92d-4b02-8de8-a6d40d7fa178 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:24:31.348647 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee0e7bd-a92d-4b02-8de8-a6d40d7fa178 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:24:31.350252 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee0e7bd-a92d-4b02-8de8-a6d40d7fa178 service nova] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:24:31.361046 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1288/2594] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:24:31 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:24:33.794628 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94dedd-1bde-4dd8-9c54-eaa85960ffd4 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:24:33.794628 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94dedd-1bde-4dd8-9c54-eaa85960ffd4 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:24:33.794628 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94dedd-1bde-4dd8-9c54-eaa85960ffd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:24:33.813731 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-4f94dedd-1bde-4dd8-9c54-eaa85960ffd4 None None] MFA Rules not processed for user `da44ae3a4e8b41d3b8a3193db939e82d`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:24:33.815241 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-4f94dedd-1bde-4dd8-9c54-eaa85960ffd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:24:33.824630 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1307/2595] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:24:33 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:24:41.356669 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=80,keystone:INSERT=2 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:24:43.807933 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:INSERT=3,keystone:SELECT=69 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:25:36.492907 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee0e7bd-a92d-4b02-8de8-a6d40d7fa178 service nova] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:25:36.495341 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-3ee0e7bd-a92d-4b02-8de8-a6d40d7fa178 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:25:36.564139 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee0e7bd-a92d-4b02-8de8-a6d40d7fa178 service nova] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:25:36.569253 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b45230-84c4-4a28-a458-65f7f8ed9beb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29653ba1a2a74232972ef7a9000518b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:25:36.569862 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b45230-84c4-4a28-a458-65f7f8ed9beb service placement] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:25:36.570143 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b45230-84c4-4a28-a458-65f7f8ed9beb service placement] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:25:36.570415 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b45230-84c4-4a28-a458-65f7f8ed9beb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:25:36.572278 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-a0b45230-84c4-4a28-a458-65f7f8ed9beb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:25:36.583599 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b45230-84c4-4a28-a458-65f7f8ed9beb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:25:36.585183 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b45230-84c4-4a28-a458-65f7f8ed9beb service placement] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:25:36.594966 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1289/2596] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:25:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:25:46.590882 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=32 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:25:48.088041 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f94dedd-1bde-4dd8-9c54-eaa85960ffd4 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:25:48.089604 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-4f94dedd-1bde-4dd8-9c54-eaa85960ffd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:25:48.105744 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f94dedd-1bde-4dd8-9c54-eaa85960ffd4 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:25:48.112008 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c152a345-b689-4b88-a56d-16aebda09cb7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55723c756cae4e7ab636054db48de02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:25:48.113034 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152a345-b689-4b88-a56d-16aebda09cb7 service nova] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:25:48.113482 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152a345-b689-4b88-a56d-16aebda09cb7 service nova] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:25:48.113933 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152a345-b689-4b88-a56d-16aebda09cb7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:25:48.116616 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-c152a345-b689-4b88-a56d-16aebda09cb7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:25:48.133536 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c152a345-b689-4b88-a56d-16aebda09cb7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:25:48.136118 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c152a345-b689-4b88-a56d-16aebda09cb7 service nova] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:25:48.150555 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1308/2597] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:25:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:25:49.646882 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b45230-84c4-4a28-a458-65f7f8ed9beb service placement] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:25:49.659082 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b45230-84c4-4a28-a458-65f7f8ed9beb service placement] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:25:49.664498 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30161854-2ad8-415b-8220-ea884f91a65b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55723c756cae4e7ab636054db48de02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:25:49.665141 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30161854-2ad8-415b-8220-ea884f91a65b service nova] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:25:49.665428 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30161854-2ad8-415b-8220-ea884f91a65b service nova] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:25:49.665701 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30161854-2ad8-415b-8220-ea884f91a65b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:25:49.667573 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-30161854-2ad8-415b-8220-ea884f91a65b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:25:49.745301 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30161854-2ad8-415b-8220-ea884f91a65b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:25:49.746784 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30161854-2ad8-415b-8220-ea884f91a65b service nova] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:25:49.765548 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1290/2598] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:25:49 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:25:57.899757 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e105fb0-9788-4d83-bd4a-9957b739fedc None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:25:57.900326 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e105fb0-9788-4d83-bd4a-9957b739fedc None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:25:57.900326 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e105fb0-9788-4d83-bd4a-9957b739fedc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:25:57.920146 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-2e105fb0-9788-4d83-bd4a-9957b739fedc None None] MFA Rules not processed for user `c5796bbaaf1345b6867d3cd5082f75b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:25:57.921274 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-2e105fb0-9788-4d83-bd4a-9957b739fedc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:25:57.978395 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1309/2599] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:25:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:25:57.990733 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30161854-2ad8-415b-8220-ea884f91a65b service nova] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:25:58.001977 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30161854-2ad8-415b-8220-ea884f91a65b service nova] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:25:58.007487 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8570f36f-5308-4e7c-845a-fd1ec0878d43 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55723c756cae4e7ab636054db48de02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:25:58.008090 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570f36f-5308-4e7c-845a-fd1ec0878d43 service nova] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:25:58.008373 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570f36f-5308-4e7c-845a-fd1ec0878d43 service nova] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:25:58.008654 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570f36f-5308-4e7c-845a-fd1ec0878d43 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:25:58.010390 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-8570f36f-5308-4e7c-845a-fd1ec0878d43 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:25:58.020846 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8570f36f-5308-4e7c-845a-fd1ec0878d43 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:25:58.022432 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8570f36f-5308-4e7c-845a-fd1ec0878d43 service nova] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:25:58.032172 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1291/2600] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:25:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:25:58.452426 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e105fb0-9788-4d83-bd4a-9957b739fedc None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:25:58.462492 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e105fb0-9788-4d83-bd4a-9957b739fedc None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:25:58.466866 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d872442b-e99b-4017-83b4-6e2592b35211 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:25:58.467525 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d872442b-e99b-4017-83b4-6e2592b35211 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:25:58.467801 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d872442b-e99b-4017-83b4-6e2592b35211 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:25:58.468230 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d872442b-e99b-4017-83b4-6e2592b35211 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:25:58.469859 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-d872442b-e99b-4017-83b4-6e2592b35211 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:25:58.480777 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d872442b-e99b-4017-83b4-6e2592b35211 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:25:58.482208 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d872442b-e99b-4017-83b4-6e2592b35211 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:25:58.492137 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1310/2601] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:25:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:26:04.455009 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8570f36f-5308-4e7c-845a-fd1ec0878d43 service nova] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:26:04.456525 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-8570f36f-5308-4e7c-845a-fd1ec0878d43 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:26:04.468463 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8570f36f-5308-4e7c-845a-fd1ec0878d43 service nova] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:26:04.473798 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392c568a-13d4-43b3-8162-173b78437c2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:26:04.473798 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c568a-13d4-43b3-8162-173b78437c2f service neutron] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:26:04.473798 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c568a-13d4-43b3-8162-173b78437c2f service neutron] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:26:04.473798 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c568a-13d4-43b3-8162-173b78437c2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:26:04.475027 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-392c568a-13d4-43b3-8162-173b78437c2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:26:04.488176 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392c568a-13d4-43b3-8162-173b78437c2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:26:04.489668 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392c568a-13d4-43b3-8162-173b78437c2f service neutron] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:26:04.500560 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1292/2602] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:26:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:26:08.187033 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d872442b-e99b-4017-83b4-6e2592b35211 service neutron] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:26:08.188746 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-d872442b-e99b-4017-83b4-6e2592b35211 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:26:08.204200 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d872442b-e99b-4017-83b4-6e2592b35211 service neutron] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:26:08.209539 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636010da-59a9-42ab-a915-35f6fe9f59c4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29653ba1a2a74232972ef7a9000518b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:26:08.210150 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636010da-59a9-42ab-a915-35f6fe9f59c4 service placement] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:26:08.210430 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636010da-59a9-42ab-a915-35f6fe9f59c4 service placement] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:26:08.211035 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636010da-59a9-42ab-a915-35f6fe9f59c4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:26:08.213446 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-636010da-59a9-42ab-a915-35f6fe9f59c4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:26:08.225995 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636010da-59a9-42ab-a915-35f6fe9f59c4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:26:08.227640 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636010da-59a9-42ab-a915-35f6fe9f59c4 service placement] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:26:08.237781 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1311/2603] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:26:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:26:11.664455 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392c568a-13d4-43b3-8162-173b78437c2f service neutron] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:26:11.664455 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-392c568a-13d4-43b3-8162-173b78437c2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:26:11.674519 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392c568a-13d4-43b3-8162-173b78437c2f service neutron] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:26:11.679169 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf63a05c-1e97-4ffa-ae63-f0b572e80fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:26:11.679800 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf63a05c-1e97-4ffa-ae63-f0b572e80fc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:26:11.680084 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf63a05c-1e97-4ffa-ae63-f0b572e80fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:26:11.680358 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf63a05c-1e97-4ffa-ae63-f0b572e80fc4 admin admin] PATH_INFO: `/v3/users/04ab53c336934355a0000bda59fa7e7c` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:26:11.682122 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf63a05c-1e97-4ffa-ae63-f0b572e80fc4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04ab53c336934355a0000bda59fa7e7c)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:26:11.684089 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf63a05c-1e97-4ffa-ae63-f0b572e80fc4 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:26:11.817070 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-bf63a05c-1e97-4ffa-ae63-f0b572e80fc4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04ab53c336934355a0000bda59fa7e7c'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:26:11.828907 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-bf63a05c-1e97-4ffa-ae63-f0b572e80fc4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04ab53c336934355a0000bda59fa7e7c'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:26:11.829654 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-bf63a05c-1e97-4ffa-ae63-f0b572e80fc4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04ab53c336934355a0000bda59fa7e7c'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:26:11.874334 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-bf63a05c-1e97-4ffa-ae63-f0b572e80fc4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04ab53c336934355a0000bda59fa7e7c'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:26:11.874967 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-bf63a05c-1e97-4ffa-ae63-f0b572e80fc4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04ab53c336934355a0000bda59fa7e7c'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:26:11.887502 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1293/2604] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:26:11 2022] DELETE /identity/v3/users/04ab53c336934355a0000bda59fa7e7c => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:26:12.162239 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636010da-59a9-42ab-a915-35f6fe9f59c4 service placement] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:26:12.178959 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636010da-59a9-42ab-a915-35f6fe9f59c4 service placement] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:26:12.252730 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ebf5be-474f-4263-b714-270d16c0600f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:26:12.253459 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ebf5be-474f-4263-b714-270d16c0600f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:26:12.253811 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ebf5be-474f-4263-b714-270d16c0600f admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:26:12.254160 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ebf5be-474f-4263-b714-270d16c0600f admin admin] PATH_INFO: `/v3/projects/e976b3d87de347818a3c62fa4806db98` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:26:12.255512 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ebf5be-474f-4263-b714-270d16c0600f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e976b3d87de347818a3c62fa4806db98)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:26:12.257725 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ebf5be-474f-4263-b714-270d16c0600f admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:26:12.378735 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-37ebf5be-474f-4263-b714-270d16c0600f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e976b3d87de347818a3c62fa4806db98', 'request_id': 'req-37ebf5be-474f-4263-b714-270d16c0600f'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:26:12.385868 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-37ebf5be-474f-4263-b714-270d16c0600f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e976b3d87de347818a3c62fa4806db98', 'request_id': 'req-37ebf5be-474f-4263-b714-270d16c0600f'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:26:12.475275 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-37ebf5be-474f-4263-b714-270d16c0600f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e976b3d87de347818a3c62fa4806db98', 'request_id': 'req-37ebf5be-474f-4263-b714-270d16c0600f'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:26:12.481515 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:26:12.488416 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-37ebf5be-474f-4263-b714-270d16c0600f admin admin] The token cache is being invalidate because project e976b3d87de347818a3c62fa4806db98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:26:12.488707 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-37ebf5be-474f-4263-b714-270d16c0600f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:26:12.489370 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-37ebf5be-474f-4263-b714-270d16c0600f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:26:12.490913 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1312/2605] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:26:12 2022] DELETE /identity/v3/projects/e976b3d87de347818a3c62fa4806db98 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:26:12.495042 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf63a05c-1e97-4ffa-ae63-f0b572e80fc4 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:26:12.496766 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-bf63a05c-1e97-4ffa-ae63-f0b572e80fc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:26:12.510605 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf63a05c-1e97-4ffa-ae63-f0b572e80fc4 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:26:12.522339 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:26:12.568808 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a2d4a8-c29b-4f09-bfbe-ffc6ba94cda3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:26:12.569364 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a2d4a8-c29b-4f09-bfbe-ffc6ba94cda3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:26:12.569643 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a2d4a8-c29b-4f09-bfbe-ffc6ba94cda3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:26:12.569911 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a2d4a8-c29b-4f09-bfbe-ffc6ba94cda3 admin admin] PATH_INFO: `/v3/users/6aec60a6c3954bb0ac7375f01a5f84ef` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:26:12.584495 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a2d4a8-c29b-4f09-bfbe-ffc6ba94cda3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6aec60a6c3954bb0ac7375f01a5f84ef)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:26:12.588933 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a2d4a8-c29b-4f09-bfbe-ffc6ba94cda3 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:26:12.746233 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-14a2d4a8-c29b-4f09-bfbe-ffc6ba94cda3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6aec60a6c3954bb0ac7375f01a5f84ef'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:26:12.757858 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-14a2d4a8-c29b-4f09-bfbe-ffc6ba94cda3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6aec60a6c3954bb0ac7375f01a5f84ef'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:26:12.758554 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-14a2d4a8-c29b-4f09-bfbe-ffc6ba94cda3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6aec60a6c3954bb0ac7375f01a5f84ef'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:26:12.806133 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-14a2d4a8-c29b-4f09-bfbe-ffc6ba94cda3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6aec60a6c3954bb0ac7375f01a5f84ef'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:26:12.807439 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-14a2d4a8-c29b-4f09-bfbe-ffc6ba94cda3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6aec60a6c3954bb0ac7375f01a5f84ef'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:26:12.821076 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1294/2606] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:26:12 2022] DELETE /identity/v3/users/6aec60a6c3954bb0ac7375f01a5f84ef => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:26:13.048142 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ebf5be-474f-4263-b714-270d16c0600f admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:26:13.049630 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-37ebf5be-474f-4263-b714-270d16c0600f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:26:13.064538 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ebf5be-474f-4263-b714-270d16c0600f admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:26:13.128813 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e78ef07-da2e-4ba1-ad53-27f2990f2932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:26:13.129632 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e78ef07-da2e-4ba1-ad53-27f2990f2932 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:26:13.129759 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e78ef07-da2e-4ba1-ad53-27f2990f2932 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:26:13.129999 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e78ef07-da2e-4ba1-ad53-27f2990f2932 admin admin] PATH_INFO: `/v3/projects/62071b8ec9c24c8d97dd6e14e677fdca` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:26:13.141818 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e78ef07-da2e-4ba1-ad53-27f2990f2932 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62071b8ec9c24c8d97dd6e14e677fdca)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:26:13.143896 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e78ef07-da2e-4ba1-ad53-27f2990f2932 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:26:13.251497 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-9e78ef07-da2e-4ba1-ad53-27f2990f2932 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62071b8ec9c24c8d97dd6e14e677fdca', 'request_id': 'req-9e78ef07-da2e-4ba1-ad53-27f2990f2932'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:26:13.258783 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-9e78ef07-da2e-4ba1-ad53-27f2990f2932 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62071b8ec9c24c8d97dd6e14e677fdca', 'request_id': 'req-9e78ef07-da2e-4ba1-ad53-27f2990f2932'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:26:13.360956 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-9e78ef07-da2e-4ba1-ad53-27f2990f2932 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62071b8ec9c24c8d97dd6e14e677fdca', 'request_id': 'req-9e78ef07-da2e-4ba1-ad53-27f2990f2932'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:26:13.374629 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-9e78ef07-da2e-4ba1-ad53-27f2990f2932 admin admin] The token cache is being invalidate because project 62071b8ec9c24c8d97dd6e14e677fdca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:26:13.375085 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-9e78ef07-da2e-4ba1-ad53-27f2990f2932 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:26:13.375847 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-9e78ef07-da2e-4ba1-ad53-27f2990f2932 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:26:13.384665 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1313/2607] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:26:13 2022] DELETE /identity/v3/projects/62071b8ec9c24c8d97dd6e14e677fdca => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:26:22.817517 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=51,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:26:23.371006 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:26:29.642813 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77139ce8-0544-4dd8-b24c-d027e0b815a7 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:26:29.643329 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77139ce8-0544-4dd8-b24c-d027e0b815a7 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:26:29.643435 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77139ce8-0544-4dd8-b24c-d027e0b815a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:26:29.662320 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-77139ce8-0544-4dd8-b24c-d027e0b815a7 None None] MFA Rules not processed for user `da44ae3a4e8b41d3b8a3193db939e82d`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:26:29.663588 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-77139ce8-0544-4dd8-b24c-d027e0b815a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:26:29.719248 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1295/2608] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:26:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:26:30.029001 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e78ef07-da2e-4ba1-ad53-27f2990f2932 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:26:30.031217 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-9e78ef07-da2e-4ba1-ad53-27f2990f2932 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:26:30.088906 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e78ef07-da2e-4ba1-ad53-27f2990f2932 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:26:30.093298 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6faee9-36ad-4507-b701-30c7c7932cbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:26:30.093989 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6faee9-36ad-4507-b701-30c7c7932cbb service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:26:30.094405 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6faee9-36ad-4507-b701-30c7c7932cbb service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:26:30.094686 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6faee9-36ad-4507-b701-30c7c7932cbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:26:30.096418 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-9f6faee9-36ad-4507-b701-30c7c7932cbb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:26:30.106072 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6faee9-36ad-4507-b701-30c7c7932cbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:26:30.107513 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6faee9-36ad-4507-b701-30c7c7932cbb service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:26:30.116634 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1314/2609] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:26:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:26:39.712694 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=25 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:26:40.113110 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=26 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:27:29.869005 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77139ce8-0544-4dd8-b24c-d027e0b815a7 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:27:29.870428 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-77139ce8-0544-4dd8-b24c-d027e0b815a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:27:29.948318 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77139ce8-0544-4dd8-b24c-d027e0b815a7 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:27:29.952073 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01ea548-413f-4015-ba16-75603b3f61e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:27:29.952674 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01ea548-413f-4015-ba16-75603b3f61e3 service cinder] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:27:29.952957 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01ea548-413f-4015-ba16-75603b3f61e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:27:29.953231 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01ea548-413f-4015-ba16-75603b3f61e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:27:29.955190 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-b01ea548-413f-4015-ba16-75603b3f61e3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:27:30.026148 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01ea548-413f-4015-ba16-75603b3f61e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:27:30.027560 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01ea548-413f-4015-ba16-75603b3f61e3 service cinder] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:27:30.038865 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1296/2610] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:27:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:27:39.852553 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6faee9-36ad-4507-b701-30c7c7932cbb service neutron] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:27:39.853936 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-9f6faee9-36ad-4507-b701-30c7c7932cbb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:27:39.866145 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6faee9-36ad-4507-b701-30c7c7932cbb service neutron] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:27:39.869814 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e54ed8-b65c-4441-9027-a312c69e0f8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:27:39.870436 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e54ed8-b65c-4441-9027-a312c69e0f8e service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:27:39.870718 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e54ed8-b65c-4441-9027-a312c69e0f8e service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:27:39.871031 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e54ed8-b65c-4441-9027-a312c69e0f8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:27:39.872702 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-c7e54ed8-b65c-4441-9027-a312c69e0f8e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:27:39.886570 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e54ed8-b65c-4441-9027-a312c69e0f8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:27:39.888052 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e54ed8-b65c-4441-9027-a312c69e0f8e service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:27:39.898871 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1315/2611] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:27:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:27:40.034041 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=63 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:27:49.894083 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=6 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:28:02.487915 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01ea548-413f-4015-ba16-75603b3f61e3 service cinder] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:02.490106 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-b01ea548-413f-4015-ba16-75603b3f61e3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:28:02.501569 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01ea548-413f-4015-ba16-75603b3f61e3 service cinder] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:02.505361 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc985331-e122-498b-9fb2-32d9a832470e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:02.506250 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc985331-e122-498b-9fb2-32d9a832470e service neutron] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:02.506955 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc985331-e122-498b-9fb2-32d9a832470e service neutron] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:02.507589 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc985331-e122-498b-9fb2-32d9a832470e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:02.509565 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-fc985331-e122-498b-9fb2-32d9a832470e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:28:02.579567 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc985331-e122-498b-9fb2-32d9a832470e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:02.581022 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc985331-e122-498b-9fb2-32d9a832470e service neutron] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:02.592521 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1297/2612] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:28:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:28:06.333247 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e54ed8-b65c-4441-9027-a312c69e0f8e service cinder] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:06.334877 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-c7e54ed8-b65c-4441-9027-a312c69e0f8e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:28:06.408576 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e54ed8-b65c-4441-9027-a312c69e0f8e service cinder] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:06.414207 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5367682d-4815-49d3-ad69-87a9e8099dab service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29653ba1a2a74232972ef7a9000518b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:06.415221 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5367682d-4815-49d3-ad69-87a9e8099dab service placement] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:06.415506 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5367682d-4815-49d3-ad69-87a9e8099dab service placement] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:06.416109 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5367682d-4815-49d3-ad69-87a9e8099dab service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:06.417915 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-5367682d-4815-49d3-ad69-87a9e8099dab service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:28:06.430558 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5367682d-4815-49d3-ad69-87a9e8099dab service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:06.432218 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5367682d-4815-49d3-ad69-87a9e8099dab service placement] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:06.442592 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1316/2613] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:28:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:28:06.491079 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc985331-e122-498b-9fb2-32d9a832470e service neutron] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:06.492765 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-fc985331-e122-498b-9fb2-32d9a832470e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:28:06.505676 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc985331-e122-498b-9fb2-32d9a832470e service neutron] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:06.511412 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55facab5-8995-4fd5-b97e-5753a44d0f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:06.512140 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55facab5-8995-4fd5-b97e-5753a44d0f97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:06.512426 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55facab5-8995-4fd5-b97e-5753a44d0f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:06.512702 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55facab5-8995-4fd5-b97e-5753a44d0f97 admin admin] PATH_INFO: `/v3/users/c5796bbaaf1345b6867d3cd5082f75b0` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:06.513818 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55facab5-8995-4fd5-b97e-5753a44d0f97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5796bbaaf1345b6867d3cd5082f75b0)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:06.515805 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55facab5-8995-4fd5-b97e-5753a44d0f97 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:06.702770 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-55facab5-8995-4fd5-b97e-5753a44d0f97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5796bbaaf1345b6867d3cd5082f75b0'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:06.715796 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-55facab5-8995-4fd5-b97e-5753a44d0f97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5796bbaaf1345b6867d3cd5082f75b0'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:06.716459 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-55facab5-8995-4fd5-b97e-5753a44d0f97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5796bbaaf1345b6867d3cd5082f75b0'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:06.775388 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-55facab5-8995-4fd5-b97e-5753a44d0f97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5796bbaaf1345b6867d3cd5082f75b0'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:06.776160 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-55facab5-8995-4fd5-b97e-5753a44d0f97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5796bbaaf1345b6867d3cd5082f75b0'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:06.789957 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1298/2614] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:28:06 2022] DELETE /identity/v3/users/c5796bbaaf1345b6867d3cd5082f75b0 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:28:07.028675 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5367682d-4815-49d3-ad69-87a9e8099dab service placement] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:07.039948 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5367682d-4815-49d3-ad69-87a9e8099dab service placement] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:07.105246 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0840892e-b764-4ede-9b1a-ad560c472aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:07.105869 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0840892e-b764-4ede-9b1a-ad560c472aee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:07.106150 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0840892e-b764-4ede-9b1a-ad560c472aee admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:07.106423 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0840892e-b764-4ede-9b1a-ad560c472aee admin admin] PATH_INFO: `/v3/projects/b531205569ec466f8e1ce8ce05427663` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:07.107571 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0840892e-b764-4ede-9b1a-ad560c472aee admin admin] RBAC: Authorizing `identity:delete_project(project_id=b531205569ec466f8e1ce8ce05427663)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:07.109455 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0840892e-b764-4ede-9b1a-ad560c472aee admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:07.230494 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-0840892e-b764-4ede-9b1a-ad560c472aee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b531205569ec466f8e1ce8ce05427663', 'request_id': 'req-0840892e-b764-4ede-9b1a-ad560c472aee'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:07.236559 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-0840892e-b764-4ede-9b1a-ad560c472aee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b531205569ec466f8e1ce8ce05427663', 'request_id': 'req-0840892e-b764-4ede-9b1a-ad560c472aee'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:07.306124 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-0840892e-b764-4ede-9b1a-ad560c472aee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b531205569ec466f8e1ce8ce05427663', 'request_id': 'req-0840892e-b764-4ede-9b1a-ad560c472aee'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:07.317046 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-0840892e-b764-4ede-9b1a-ad560c472aee admin admin] The token cache is being invalidate because project b531205569ec466f8e1ce8ce05427663 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:28:07.317336 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-0840892e-b764-4ede-9b1a-ad560c472aee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:07.317965 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-0840892e-b764-4ede-9b1a-ad560c472aee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:07.320083 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1317/2615] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:28:07 2022] DELETE /identity/v3/projects/b531205569ec466f8e1ce8ce05427663 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:28:07.325210 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f5299-273a-48a2-b91f-b2cda697ef93 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:07.325529 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f5299-273a-48a2-b91f-b2cda697ef93 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:07.325836 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f5299-273a-48a2-b91f-b2cda697ef93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:07.346428 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-627f5299-273a-48a2-b91f-b2cda697ef93 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:28:07.347809 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-627f5299-273a-48a2-b91f-b2cda697ef93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:28:07.406572 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1299/2616] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:28:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:28:07.414460 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f59396-89fd-405a-8f3a-03ab875a0980 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:07.414970 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f59396-89fd-405a-8f3a-03ab875a0980 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:07.415085 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f59396-89fd-405a-8f3a-03ab875a0980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:07.434186 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-e1f59396-89fd-405a-8f3a-03ab875a0980 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:28:07.435670 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-e1f59396-89fd-405a-8f3a-03ab875a0980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:28:07.444116 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1318/2617] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:28:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:28:07.448337 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627f5299-273a-48a2-b91f-b2cda697ef93 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:07.457980 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627f5299-273a-48a2-b91f-b2cda697ef93 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:07.463409 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0dd16a-2053-423e-b020-3231cc9fb77f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:07.464013 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0dd16a-2053-423e-b020-3231cc9fb77f admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:07.464296 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0dd16a-2053-423e-b020-3231cc9fb77f admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:07.464568 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0dd16a-2053-423e-b020-3231cc9fb77f admin admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:07.465234 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0dd16a-2053-423e-b020-3231cc9fb77f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:28:07.465535 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0dd16a-2053-423e-b020-3231cc9fb77f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:07.467184 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0dd16a-2053-423e-b020-3231cc9fb77f admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:07.468701 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:28:07.483369 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1300/2618] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:28:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 17:28:07.487841 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f59396-89fd-405a-8f3a-03ab875a0980 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:07.497811 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f59396-89fd-405a-8f3a-03ab875a0980 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:07.502604 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b55226-0059-4bfa-891d-21538db84af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:07.503236 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b55226-0059-4bfa-891d-21538db84af5 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:07.503516 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b55226-0059-4bfa-891d-21538db84af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:07.503787 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b55226-0059-4bfa-891d-21538db84af5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:07.504406 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b55226-0059-4bfa-891d-21538db84af5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:07.507371 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b55226-0059-4bfa-891d-21538db84af5 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:07.518337 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:28:07.622983 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1319/2619] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:28:07 2022] POST /identity/v3/projects => generated 365 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:28:07.627490 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0dd16a-2053-423e-b020-3231cc9fb77f admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:07.638169 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0dd16a-2053-423e-b020-3231cc9fb77f admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:07.695241 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567b553b-29a3-4c5d-b07f-4b116c35f352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:07.695945 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567b553b-29a3-4c5d-b07f-4b116c35f352 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:07.696219 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567b553b-29a3-4c5d-b07f-4b116c35f352 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:07.696491 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567b553b-29a3-4c5d-b07f-4b116c35f352 admin admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:07.697114 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567b553b-29a3-4c5d-b07f-4b116c35f352 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:07.699079 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567b553b-29a3-4c5d-b07f-4b116c35f352 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:07.816670 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1301/2620] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:28:07 2022] POST /identity/v3/users => generated 283 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:28:07.821491 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b55226-0059-4bfa-891d-21538db84af5 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:07.832387 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b55226-0059-4bfa-891d-21538db84af5 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:07.837736 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4e8866-6dc7-45be-84d2-b160c1b19482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:07.838213 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e8866-6dc7-45be-84d2-b160c1b19482 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:07.838491 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e8866-6dc7-45be-84d2-b160c1b19482 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:07.838759 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e8866-6dc7-45be-84d2-b160c1b19482 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:07.839545 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4e8866-6dc7-45be-84d2-b160c1b19482 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:07.841628 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4e8866-6dc7-45be-84d2-b160c1b19482 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:07.853443 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1320/2621] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:28:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:28:07.856702 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567b553b-29a3-4c5d-b07f-4b116c35f352 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:07.867392 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567b553b-29a3-4c5d-b07f-4b116c35f352 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:07.872083 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd15810-78ab-4003-b58b-699063006c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:07.876197 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd15810-78ab-4003-b58b-699063006c29 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:07.876197 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd15810-78ab-4003-b58b-699063006c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:07.876197 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd15810-78ab-4003-b58b-699063006c29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:07.876197 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd15810-78ab-4003-b58b-699063006c29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:07.876197 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd15810-78ab-4003-b58b-699063006c29 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:07.886606 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1302/2622] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:28:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:28:07.890711 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4e8866-6dc7-45be-84d2-b160c1b19482 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:07.901135 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4e8866-6dc7-45be-84d2-b160c1b19482 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:07.905697 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad0ff6c-e27e-4f90-8617-650577d4a9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:07.906366 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad0ff6c-e27e-4f90-8617-650577d4a9b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:07.906645 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad0ff6c-e27e-4f90-8617-650577d4a9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:07.906944 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad0ff6c-e27e-4f90-8617-650577d4a9b3 admin admin] PATH_INFO: `/v3/projects/05f3d59a02ae45a3b7aae6652c200162/users/9f4c36cbdfe04199b78d78c9629a698f/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:07.922264 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad0ff6c-e27e-4f90-8617-650577d4a9b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05f3d59a02ae45a3b7aae6652c200162, user_id=9f4c36cbdfe04199b78d78c9629a698f, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:07.924919 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad0ff6c-e27e-4f90-8617-650577d4a9b3 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:07.986287 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1321/2623] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:28:07 2022] PUT /identity/v3/projects/05f3d59a02ae45a3b7aae6652c200162/users/9f4c36cbdfe04199b78d78c9629a698f/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:28:07.997675 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a158d70-2908-4dbc-8f46-94efc01a9cb6 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:07.997959 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a158d70-2908-4dbc-8f46-94efc01a9cb6 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:07.998230 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a158d70-2908-4dbc-8f46-94efc01a9cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:08.027608 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-4a158d70-2908-4dbc-8f46-94efc01a9cb6 None None] MFA Rules not processed for user `9f4c36cbdfe04199b78d78c9629a698f`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:28:08.028818 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-4a158d70-2908-4dbc-8f46-94efc01a9cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:28:08.092621 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1303/2624] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:28:07 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:28:08.104123 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad0ff6c-e27e-4f90-8617-650577d4a9b3 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:08.105809 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-6ad0ff6c-e27e-4f90-8617-650577d4a9b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:28:08.166728 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad0ff6c-e27e-4f90-8617-650577d4a9b3 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:08.171800 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7119ccc9-3c5f-48a3-bbf6-7d0aaf5dbdd8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:08.172476 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7119ccc9-3c5f-48a3-bbf6-7d0aaf5dbdd8 service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:08.172771 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7119ccc9-3c5f-48a3-bbf6-7d0aaf5dbdd8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:08.173035 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7119ccc9-3c5f-48a3-bbf6-7d0aaf5dbdd8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:08.183750 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7119ccc9-3c5f-48a3-bbf6-7d0aaf5dbdd8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:08.185468 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7119ccc9-3c5f-48a3-bbf6-7d0aaf5dbdd8 service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:08.194857 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1322/2625] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:28:08 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:28:14.235704 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36544de-ae52-4c2f-b7ba-76959797d783 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:14.236226 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36544de-ae52-4c2f-b7ba-76959797d783 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:14.236356 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36544de-ae52-4c2f-b7ba-76959797d783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:14.255486 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-f36544de-ae52-4c2f-b7ba-76959797d783 None None] MFA Rules not processed for user `da44ae3a4e8b41d3b8a3193db939e82d`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:28:14.256753 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-f36544de-ae52-4c2f-b7ba-76959797d783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:28:14.321031 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1304/2626] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:28:14 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:28:17.883403 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7119ccc9-3c5f-48a3-bbf6-7d0aaf5dbdd8 service cinder] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:17.952928 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7119ccc9-3c5f-48a3-bbf6-7d0aaf5dbdd8 service cinder] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:17.957804 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5949b733-433f-42e4-a412-15d633b09fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:17.959332 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5949b733-433f-42e4-a412-15d633b09fcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:17.959616 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5949b733-433f-42e4-a412-15d633b09fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:17.959887 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5949b733-433f-42e4-a412-15d633b09fcf admin admin] PATH_INFO: `/v3/users/9f4c36cbdfe04199b78d78c9629a698f` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:17.961155 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5949b733-433f-42e4-a412-15d633b09fcf admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f4c36cbdfe04199b78d78c9629a698f)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:17.962999 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5949b733-433f-42e4-a412-15d633b09fcf admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:17.983354 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36544de-ae52-4c2f-b7ba-76959797d783 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:17.983354 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-f36544de-ae52-4c2f-b7ba-76959797d783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:28:17.991501 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:28:18.039102 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36544de-ae52-4c2f-b7ba-76959797d783 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:18.043210 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1d586a-6b28-4b36-af35-802800fd75db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:18.044032 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1d586a-6b28-4b36-af35-802800fd75db service neutron] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:18.044724 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1d586a-6b28-4b36-af35-802800fd75db service neutron] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:18.045055 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1d586a-6b28-4b36-af35-802800fd75db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:18.046826 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-dc1d586a-6b28-4b36-af35-802800fd75db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:28:18.057217 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1d586a-6b28-4b36-af35-802800fd75db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:18.058640 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1d586a-6b28-4b36-af35-802800fd75db service neutron] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:18.066676 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1305/2627] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:28:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 17:28:18.117509 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-5949b733-433f-42e4-a412-15d633b09fcf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f4c36cbdfe04199b78d78c9629a698f'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:18.214095 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-5949b733-433f-42e4-a412-15d633b09fcf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f4c36cbdfe04199b78d78c9629a698f'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:18.214821 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-5949b733-433f-42e4-a412-15d633b09fcf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f4c36cbdfe04199b78d78c9629a698f'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:18.215557 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-5949b733-433f-42e4-a412-15d633b09fcf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f4c36cbdfe04199b78d78c9629a698f'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:18.221452 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:28:18.223594 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-5949b733-433f-42e4-a412-15d633b09fcf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f4c36cbdfe04199b78d78c9629a698f'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:18.240894 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1323/2628] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:28:17 2022] DELETE /identity/v3/users/9f4c36cbdfe04199b78d78c9629a698f => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:28:18.245506 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8673cc-7238-41d4-b985-2b5ef14d1f3d None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:18.245900 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8673cc-7238-41d4-b985-2b5ef14d1f3d None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:18.246235 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8673cc-7238-41d4-b985-2b5ef14d1f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:18.267052 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-dd8673cc-7238-41d4-b985-2b5ef14d1f3d None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:28:18.268337 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-dd8673cc-7238-41d4-b985-2b5ef14d1f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:28:18.329565 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1306/2629] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:28:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 17:28:18.339638 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5949b733-433f-42e4-a412-15d633b09fcf admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:18.345510 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-5949b733-433f-42e4-a412-15d633b09fcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:28:18.404056 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5949b733-433f-42e4-a412-15d633b09fcf admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:18.407700 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c2cab-3b0b-4d49-8723-549215824fc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:18.408309 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c2cab-3b0b-4d49-8723-549215824fc8 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:18.408590 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c2cab-3b0b-4d49-8723-549215824fc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:18.408861 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c2cab-3b0b-4d49-8723-549215824fc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:18.419341 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712c2cab-3b0b-4d49-8723-549215824fc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:18.420801 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712c2cab-3b0b-4d49-8723-549215824fc8 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:18.431393 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1324/2630] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:28:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:28:19.022751 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8673cc-7238-41d4-b985-2b5ef14d1f3d None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:19.024281 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-dd8673cc-7238-41d4-b985-2b5ef14d1f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:28:19.047134 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8673cc-7238-41d4-b985-2b5ef14d1f3d None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:19.052798 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca5cf14-6789-406c-a942-a2b8a7181758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:19.054005 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5cf14-6789-406c-a942-a2b8a7181758 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:19.054005 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5cf14-6789-406c-a942-a2b8a7181758 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:19.054178 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5cf14-6789-406c-a942-a2b8a7181758 admin admin] PATH_INFO: `/v3/projects/05f3d59a02ae45a3b7aae6652c200162` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:19.055287 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca5cf14-6789-406c-a942-a2b8a7181758 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05f3d59a02ae45a3b7aae6652c200162)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:19.057297 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca5cf14-6789-406c-a942-a2b8a7181758 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:19.170392 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-8ca5cf14-6789-406c-a942-a2b8a7181758 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05f3d59a02ae45a3b7aae6652c200162', 'request_id': 'req-8ca5cf14-6789-406c-a942-a2b8a7181758'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:19.226470 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-8ca5cf14-6789-406c-a942-a2b8a7181758 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05f3d59a02ae45a3b7aae6652c200162', 'request_id': 'req-8ca5cf14-6789-406c-a942-a2b8a7181758'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:19.232871 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-8ca5cf14-6789-406c-a942-a2b8a7181758 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05f3d59a02ae45a3b7aae6652c200162', 'request_id': 'req-8ca5cf14-6789-406c-a942-a2b8a7181758'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:19.244780 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-8ca5cf14-6789-406c-a942-a2b8a7181758 admin admin] The token cache is being invalidate because project 05f3d59a02ae45a3b7aae6652c200162 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81650) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:28:19.245069 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-8ca5cf14-6789-406c-a942-a2b8a7181758 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:19.245738 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-8ca5cf14-6789-406c-a942-a2b8a7181758 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:19.247249 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1307/2631] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:28:19 2022] DELETE /identity/v3/projects/05f3d59a02ae45a3b7aae6652c200162 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:28:19.252287 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0333b7-78a6-443a-9296-ffb452f71cb2 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:19.252763 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0333b7-78a6-443a-9296-ffb452f71cb2 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:19.252869 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0333b7-78a6-443a-9296-ffb452f71cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:19.273469 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-cc0333b7-78a6-443a-9296-ffb452f71cb2 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:28:19.274666 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-cc0333b7-78a6-443a-9296-ffb452f71cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:28:19.336763 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1325/2632] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:28:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:28:19.345795 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bb63d3-eeed-4f19-b9c6-3ac5664749c9 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:19.346077 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bb63d3-eeed-4f19-b9c6-3ac5664749c9 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:19.346349 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bb63d3-eeed-4f19-b9c6-3ac5664749c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:19.367504 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-41bb63d3-eeed-4f19-b9c6-3ac5664749c9 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:28:19.368733 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-41bb63d3-eeed-4f19-b9c6-3ac5664749c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:28:19.376933 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1308/2633] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:28:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:28:19.381033 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0333b7-78a6-443a-9296-ffb452f71cb2 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:19.390541 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0333b7-78a6-443a-9296-ffb452f71cb2 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:19.395075 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abe62ad-1a71-4ebe-9b93-cc60d515dc30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:19.395700 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abe62ad-1a71-4ebe-9b93-cc60d515dc30 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:19.395976 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abe62ad-1a71-4ebe-9b93-cc60d515dc30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:19.396243 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abe62ad-1a71-4ebe-9b93-cc60d515dc30 admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:19.396954 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abe62ad-1a71-4ebe-9b93-cc60d515dc30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:28:19.397255 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abe62ad-1a71-4ebe-9b93-cc60d515dc30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:19.398839 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abe62ad-1a71-4ebe-9b93-cc60d515dc30 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:19.415119 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1326/2634] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:28:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 17:28:19.416154 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bb63d3-eeed-4f19-b9c6-3ac5664749c9 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:19.425525 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bb63d3-eeed-4f19-b9c6-3ac5664749c9 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:19.431716 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895d6e71-8f78-4560-8a20-6378ea144680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:19.432342 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d6e71-8f78-4560-8a20-6378ea144680 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:19.432622 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d6e71-8f78-4560-8a20-6378ea144680 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:19.432919 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d6e71-8f78-4560-8a20-6378ea144680 admin admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:19.433547 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895d6e71-8f78-4560-8a20-6378ea144680 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:19.435741 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895d6e71-8f78-4560-8a20-6378ea144680 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:19.535924 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1309/2635] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:28:19 2022] POST /identity/v3/projects => generated 405 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:28:19.540530 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abe62ad-1a71-4ebe-9b93-cc60d515dc30 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:19.550620 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abe62ad-1a71-4ebe-9b93-cc60d515dc30 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:19.608224 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb54d2da-ff40-4d1c-b549-9450a81c75ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:19.608835 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54d2da-ff40-4d1c-b549-9450a81c75ff admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:19.609118 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54d2da-ff40-4d1c-b549-9450a81c75ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:19.611800 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54d2da-ff40-4d1c-b549-9450a81c75ff admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:19.611800 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb54d2da-ff40-4d1c-b549-9450a81c75ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:19.612068 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb54d2da-ff40-4d1c-b549-9450a81c75ff admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:19.677593 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:28:19.687327 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1327/2636] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:28:19 2022] POST /identity/v3/users => generated 303 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:28:19.688701 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895d6e71-8f78-4560-8a20-6378ea144680 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:19.695668 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:28:19.699675 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895d6e71-8f78-4560-8a20-6378ea144680 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:19.704527 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d32b9-baea-4578-bb16-efd55028f542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:19.705140 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d32b9-baea-4578-bb16-efd55028f542 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:19.705420 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d32b9-baea-4578-bb16-efd55028f542 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:19.705694 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d32b9-baea-4578-bb16-efd55028f542 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:19.706340 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060d32b9-baea-4578-bb16-efd55028f542 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:19.707981 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060d32b9-baea-4578-bb16-efd55028f542 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:19.718793 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1310/2637] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:28:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:28:19.722739 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb54d2da-ff40-4d1c-b549-9450a81c75ff admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:19.734337 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb54d2da-ff40-4d1c-b549-9450a81c75ff admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:19.739028 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c181396-eb7c-4302-8099-891171a9a338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:19.740312 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c181396-eb7c-4302-8099-891171a9a338 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:19.740658 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c181396-eb7c-4302-8099-891171a9a338 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:19.740869 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c181396-eb7c-4302-8099-891171a9a338 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:19.741495 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c181396-eb7c-4302-8099-891171a9a338 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:19.743134 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c181396-eb7c-4302-8099-891171a9a338 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:19.753792 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1328/2638] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:28:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:28:19.759161 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d32b9-baea-4578-bb16-efd55028f542 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:19.769684 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d32b9-baea-4578-bb16-efd55028f542 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:19.774490 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b66d077-fe4f-4eec-af0f-dff3c74067a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:19.775201 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b66d077-fe4f-4eec-af0f-dff3c74067a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:19.778131 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b66d077-fe4f-4eec-af0f-dff3c74067a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:19.778131 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b66d077-fe4f-4eec-af0f-dff3c74067a9 admin admin] PATH_INFO: `/v3/projects/90f9acab9593467491f696b487d4795f/users/0af84f3b8acd4b2285ecdc3b9a2e6882/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:19.790357 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b66d077-fe4f-4eec-af0f-dff3c74067a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90f9acab9593467491f696b487d4795f, user_id=0af84f3b8acd4b2285ecdc3b9a2e6882, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:19.793058 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b66d077-fe4f-4eec-af0f-dff3c74067a9 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:19.923987 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1311/2639] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:28:19 2022] PUT /identity/v3/projects/90f9acab9593467491f696b487d4795f/users/0af84f3b8acd4b2285ecdc3b9a2e6882/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:28:19.929802 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbba678-e180-4aec-9d13-0aaf2cff1922 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:19.932624 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbba678-e180-4aec-9d13-0aaf2cff1922 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:19.932624 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbba678-e180-4aec-9d13-0aaf2cff1922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:19.948952 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-0cbba678-e180-4aec-9d13-0aaf2cff1922 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:28:19.950153 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-0cbba678-e180-4aec-9d13-0aaf2cff1922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:28:20.010096 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1329/2640] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:28:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:28:20.021284 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b66d077-fe4f-4eec-af0f-dff3c74067a9 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:20.023065 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-2b66d077-fe4f-4eec-af0f-dff3c74067a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:28:20.079908 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b66d077-fe4f-4eec-af0f-dff3c74067a9 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:20.083720 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c41741-74de-4b57-80a7-656152b5dace service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:20.084312 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c41741-74de-4b57-80a7-656152b5dace service neutron] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:20.084590 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c41741-74de-4b57-80a7-656152b5dace service neutron] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:20.084862 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c41741-74de-4b57-80a7-656152b5dace service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:20.093735 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c41741-74de-4b57-80a7-656152b5dace service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:20.095204 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c41741-74de-4b57-80a7-656152b5dace service neutron] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:20.105523 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1312/2641] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:28:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:28:21.330221 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3005f3e-4e25-4ea8-a06d-36695dc17c6e None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:21.330708 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3005f3e-4e25-4ea8-a06d-36695dc17c6e None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:21.330943 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3005f3e-4e25-4ea8-a06d-36695dc17c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:21.359595 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-d3005f3e-4e25-4ea8-a06d-36695dc17c6e None None] MFA Rules not processed for user `0af84f3b8acd4b2285ecdc3b9a2e6882`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:28:21.360849 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-d3005f3e-4e25-4ea8-a06d-36695dc17c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:28:21.426412 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1330/2642] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:28:21 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:28:21.431837 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c41741-74de-4b57-80a7-656152b5dace service neutron] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:21.441791 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c41741-74de-4b57-80a7-656152b5dace service neutron] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:21.446672 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4a6ce7-fd53-47a9-af0d-9e99477b0ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:21.447300 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a6ce7-fd53-47a9-af0d-9e99477b0ec1 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:21.447581 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a6ce7-fd53-47a9-af0d-9e99477b0ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:21.447854 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a6ce7-fd53-47a9-af0d-9e99477b0ec1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:21.448466 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4a6ce7-fd53-47a9-af0d-9e99477b0ec1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:21.450355 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4a6ce7-fd53-47a9-af0d-9e99477b0ec1 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:21.529576 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1313/2643] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:28:21 2022] POST /identity/v3/projects => generated 403 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:28:21.533652 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3005f3e-4e25-4ea8-a06d-36695dc17c6e None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:21.545035 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3005f3e-4e25-4ea8-a06d-36695dc17c6e None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:21.601637 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403944c5-d1fa-46c8-b0a2-47740d9c3f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:21.602236 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403944c5-d1fa-46c8-b0a2-47740d9c3f97 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:21.602515 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403944c5-d1fa-46c8-b0a2-47740d9c3f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:21.602787 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403944c5-d1fa-46c8-b0a2-47740d9c3f97 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:21.603502 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403944c5-d1fa-46c8-b0a2-47740d9c3f97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:21.605393 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403944c5-d1fa-46c8-b0a2-47740d9c3f97 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:21.694673 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1331/2644] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:28:21 2022] POST /identity/v3/users => generated 308 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:28:21.698734 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4a6ce7-fd53-47a9-af0d-9e99477b0ec1 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:21.709626 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4a6ce7-fd53-47a9-af0d-9e99477b0ec1 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:21.714135 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acb9ae1-d3b5-49a6-8ad9-a10345c34249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:21.714740 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb9ae1-d3b5-49a6-8ad9-a10345c34249 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:21.715069 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb9ae1-d3b5-49a6-8ad9-a10345c34249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:21.715343 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb9ae1-d3b5-49a6-8ad9-a10345c34249 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:21.715973 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acb9ae1-d3b5-49a6-8ad9-a10345c34249 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:21.717578 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acb9ae1-d3b5-49a6-8ad9-a10345c34249 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:21.728823 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1314/2645] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:28:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:28:21.732977 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403944c5-d1fa-46c8-b0a2-47740d9c3f97 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:21.743020 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403944c5-d1fa-46c8-b0a2-47740d9c3f97 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:21.747539 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a26c327-a40d-4356-ba06-45c15fea8ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:21.748178 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a26c327-a40d-4356-ba06-45c15fea8ef5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:21.748453 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a26c327-a40d-4356-ba06-45c15fea8ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:21.748720 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a26c327-a40d-4356-ba06-45c15fea8ef5 admin admin] PATH_INFO: `/v3/projects/b1f8ec6855ac4847810d33f7394cfd47/users/4a7fefadbc3242ffb18e5957dafd81b2/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:21.750892 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:28:21.763643 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a26c327-a40d-4356-ba06-45c15fea8ef5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1f8ec6855ac4847810d33f7394cfd47, user_id=4a7fefadbc3242ffb18e5957dafd81b2, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:21.766276 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a26c327-a40d-4356-ba06-45c15fea8ef5 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:21.831210 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1332/2646] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:28:21 2022] PUT /identity/v3/projects/b1f8ec6855ac4847810d33f7394cfd47/users/4a7fefadbc3242ffb18e5957dafd81b2/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:28:21.836669 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acb9ae1-d3b5-49a6-8ad9-a10345c34249 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:21.847144 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acb9ae1-d3b5-49a6-8ad9-a10345c34249 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:21.904662 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b12c173-de93-40ab-a755-cdb9cddbcb2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:21.905273 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b12c173-de93-40ab-a755-cdb9cddbcb2e admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:21.905558 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b12c173-de93-40ab-a755-cdb9cddbcb2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:21.905834 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b12c173-de93-40ab-a755-cdb9cddbcb2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:21.906463 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b12c173-de93-40ab-a755-cdb9cddbcb2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:21.908097 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b12c173-de93-40ab-a755-cdb9cddbcb2e admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:21.918167 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1315/2647] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:28:21 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:28:21.922304 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a26c327-a40d-4356-ba06-45c15fea8ef5 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:21.932681 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a26c327-a40d-4356-ba06-45c15fea8ef5 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:21.937403 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd01b33-7665-4f94-8e0c-b92f6cea2c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:21.938046 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd01b33-7665-4f94-8e0c-b92f6cea2c45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:21.938318 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd01b33-7665-4f94-8e0c-b92f6cea2c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:21.938583 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd01b33-7665-4f94-8e0c-b92f6cea2c45 admin admin] PATH_INFO: `/v3/projects/b1f8ec6855ac4847810d33f7394cfd47/users/4a7fefadbc3242ffb18e5957dafd81b2/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:21.940587 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd01b33-7665-4f94-8e0c-b92f6cea2c45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1f8ec6855ac4847810d33f7394cfd47, user_id=4a7fefadbc3242ffb18e5957dafd81b2, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:21.943573 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd01b33-7665-4f94-8e0c-b92f6cea2c45 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:21.953336 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1333/2648] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:28:21 2022] PUT /identity/v3/projects/b1f8ec6855ac4847810d33f7394cfd47/users/4a7fefadbc3242ffb18e5957dafd81b2/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:28:23.166534 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e10757f-c196-4a46-9bb3-b17cf8f9d6b2 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:23.167134 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e10757f-c196-4a46-9bb3-b17cf8f9d6b2 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:23.167134 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e10757f-c196-4a46-9bb3-b17cf8f9d6b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:23.196879 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-9e10757f-c196-4a46-9bb3-b17cf8f9d6b2 None None] MFA Rules not processed for user `4a7fefadbc3242ffb18e5957dafd81b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:28:23.198207 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-9e10757f-c196-4a46-9bb3-b17cf8f9d6b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:28:23.222676 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:28:23.268041 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1316/2649] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:28:23 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:28:24.530243 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd01b33-7665-4f94-8e0c-b92f6cea2c45 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:24.595602 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd01b33-7665-4f94-8e0c-b92f6cea2c45 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:24.600397 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98d1cf1-ef67-4e92-aa1f-6dfd70d76db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:24.601032 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d1cf1-ef67-4e92-aa1f-6dfd70d76db3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:24.601312 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d1cf1-ef67-4e92-aa1f-6dfd70d76db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:24.601586 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d1cf1-ef67-4e92-aa1f-6dfd70d76db3 admin admin] PATH_INFO: `/v3/users/0af84f3b8acd4b2285ecdc3b9a2e6882` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:24.602704 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98d1cf1-ef67-4e92-aa1f-6dfd70d76db3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0af84f3b8acd4b2285ecdc3b9a2e6882)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:24.604650 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98d1cf1-ef67-4e92-aa1f-6dfd70d76db3 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:24.749405 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-c98d1cf1-ef67-4e92-aa1f-6dfd70d76db3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0af84f3b8acd4b2285ecdc3b9a2e6882'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:24.814963 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-c98d1cf1-ef67-4e92-aa1f-6dfd70d76db3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0af84f3b8acd4b2285ecdc3b9a2e6882'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:24.815588 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-c98d1cf1-ef67-4e92-aa1f-6dfd70d76db3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0af84f3b8acd4b2285ecdc3b9a2e6882'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:24.816188 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-c98d1cf1-ef67-4e92-aa1f-6dfd70d76db3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0af84f3b8acd4b2285ecdc3b9a2e6882'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:24.824304 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-c98d1cf1-ef67-4e92-aa1f-6dfd70d76db3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0af84f3b8acd4b2285ecdc3b9a2e6882'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:24.841443 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1334/2650] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:28:24 2022] DELETE /identity/v3/users/0af84f3b8acd4b2285ecdc3b9a2e6882 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:28:25.084972 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e10757f-c196-4a46-9bb3-b17cf8f9d6b2 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:25.099505 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e10757f-c196-4a46-9bb3-b17cf8f9d6b2 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:25.160198 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c6e7ce-885a-4bd3-be89-28926dbf29d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:25.160841 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6e7ce-885a-4bd3-be89-28926dbf29d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:25.161797 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6e7ce-885a-4bd3-be89-28926dbf29d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:25.162076 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6e7ce-885a-4bd3-be89-28926dbf29d3 admin admin] PATH_INFO: `/v3/projects/90f9acab9593467491f696b487d4795f` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:25.163306 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c6e7ce-885a-4bd3-be89-28926dbf29d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90f9acab9593467491f696b487d4795f)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:25.165235 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c6e7ce-885a-4bd3-be89-28926dbf29d3 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:25.277578 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-00c6e7ce-885a-4bd3-be89-28926dbf29d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90f9acab9593467491f696b487d4795f', 'request_id': 'req-00c6e7ce-885a-4bd3-be89-28926dbf29d3'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:25.357189 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-00c6e7ce-885a-4bd3-be89-28926dbf29d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90f9acab9593467491f696b487d4795f', 'request_id': 'req-00c6e7ce-885a-4bd3-be89-28926dbf29d3'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:25.363768 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-00c6e7ce-885a-4bd3-be89-28926dbf29d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90f9acab9593467491f696b487d4795f', 'request_id': 'req-00c6e7ce-885a-4bd3-be89-28926dbf29d3'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:25.375784 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-00c6e7ce-885a-4bd3-be89-28926dbf29d3 admin admin] The token cache is being invalidate because project 90f9acab9593467491f696b487d4795f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81650) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:28:25.376078 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-00c6e7ce-885a-4bd3-be89-28926dbf29d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:25.376826 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-00c6e7ce-885a-4bd3-be89-28926dbf29d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:25.378235 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1317/2651] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:28:25 2022] DELETE /identity/v3/projects/90f9acab9593467491f696b487d4795f => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:28:25.383277 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98d1cf1-ef67-4e92-aa1f-6dfd70d76db3 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:25.384729 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-c98d1cf1-ef67-4e92-aa1f-6dfd70d76db3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:28:25.399052 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98d1cf1-ef67-4e92-aa1f-6dfd70d76db3 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:25.448841 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:28:25.461487 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e42ff33-5642-45a9-8db4-8a3556d9025d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:25.462210 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e42ff33-5642-45a9-8db4-8a3556d9025d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:25.462487 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e42ff33-5642-45a9-8db4-8a3556d9025d admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:25.462758 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e42ff33-5642-45a9-8db4-8a3556d9025d admin admin] PATH_INFO: `/v3/users/4a7fefadbc3242ffb18e5957dafd81b2` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:25.464109 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e42ff33-5642-45a9-8db4-8a3556d9025d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a7fefadbc3242ffb18e5957dafd81b2)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:25.466013 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e42ff33-5642-45a9-8db4-8a3556d9025d admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:25.714177 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-0e42ff33-5642-45a9-8db4-8a3556d9025d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a7fefadbc3242ffb18e5957dafd81b2'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:25.797294 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-0e42ff33-5642-45a9-8db4-8a3556d9025d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a7fefadbc3242ffb18e5957dafd81b2'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:25.797937 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-0e42ff33-5642-45a9-8db4-8a3556d9025d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a7fefadbc3242ffb18e5957dafd81b2'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:25.798530 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-0e42ff33-5642-45a9-8db4-8a3556d9025d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a7fefadbc3242ffb18e5957dafd81b2'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:25.807246 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-0e42ff33-5642-45a9-8db4-8a3556d9025d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a7fefadbc3242ffb18e5957dafd81b2'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:25.826289 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1335/2652] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:28:25 2022] DELETE /identity/v3/users/4a7fefadbc3242ffb18e5957dafd81b2 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:28:26.096446 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c6e7ce-885a-4bd3-be89-28926dbf29d3 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:26.098060 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-00c6e7ce-885a-4bd3-be89-28926dbf29d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:28:26.113318 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c6e7ce-885a-4bd3-be89-28926dbf29d3 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:26.162678 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:28:26.172363 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a06e20-c0cd-4958-be3e-7d9d82077cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:26.172999 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a06e20-c0cd-4958-be3e-7d9d82077cf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:26.173288 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a06e20-c0cd-4958-be3e-7d9d82077cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:26.173547 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a06e20-c0cd-4958-be3e-7d9d82077cf9 admin admin] PATH_INFO: `/v3/projects/b1f8ec6855ac4847810d33f7394cfd47` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:26.174661 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a06e20-c0cd-4958-be3e-7d9d82077cf9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1f8ec6855ac4847810d33f7394cfd47)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:26.177265 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a06e20-c0cd-4958-be3e-7d9d82077cf9 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:26.290484 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-13a06e20-c0cd-4958-be3e-7d9d82077cf9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1f8ec6855ac4847810d33f7394cfd47', 'request_id': 'req-13a06e20-c0cd-4958-be3e-7d9d82077cf9'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:26.357312 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-13a06e20-c0cd-4958-be3e-7d9d82077cf9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1f8ec6855ac4847810d33f7394cfd47', 'request_id': 'req-13a06e20-c0cd-4958-be3e-7d9d82077cf9'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:26.363962 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-13a06e20-c0cd-4958-be3e-7d9d82077cf9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1f8ec6855ac4847810d33f7394cfd47', 'request_id': 'req-13a06e20-c0cd-4958-be3e-7d9d82077cf9'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:26.375355 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-13a06e20-c0cd-4958-be3e-7d9d82077cf9 admin admin] The token cache is being invalidate because project b1f8ec6855ac4847810d33f7394cfd47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81650) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:28:26.375646 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-13a06e20-c0cd-4958-be3e-7d9d82077cf9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:26.376295 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-13a06e20-c0cd-4958-be3e-7d9d82077cf9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:26.377837 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1318/2653] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:28:26 2022] DELETE /identity/v3/projects/b1f8ec6855ac4847810d33f7394cfd47 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:28:26.383162 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f999895-110e-4e38-af7e-106335351909 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:26.383631 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f999895-110e-4e38-af7e-106335351909 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:26.384281 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f999895-110e-4e38-af7e-106335351909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:26.404358 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-6f999895-110e-4e38-af7e-106335351909 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:28:26.405639 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-6f999895-110e-4e38-af7e-106335351909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:28:26.464589 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1336/2654] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:28:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:28:26.473586 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a228fd72-b439-4331-99a2-de64471a70c5 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:26.473874 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a228fd72-b439-4331-99a2-de64471a70c5 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:26.474151 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a228fd72-b439-4331-99a2-de64471a70c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:26.494294 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-a228fd72-b439-4331-99a2-de64471a70c5 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:28:26.495532 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-a228fd72-b439-4331-99a2-de64471a70c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:28:26.503422 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1319/2655] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:28:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:28:26.507721 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f999895-110e-4e38-af7e-106335351909 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:26.517335 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f999895-110e-4e38-af7e-106335351909 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:26.522517 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df4603e-31dd-4eea-b336-c8291db0a057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:26.523157 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df4603e-31dd-4eea-b336-c8291db0a057 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:26.523449 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df4603e-31dd-4eea-b336-c8291db0a057 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:26.523716 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df4603e-31dd-4eea-b336-c8291db0a057 admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:26.524356 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df4603e-31dd-4eea-b336-c8291db0a057 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:28:26.524656 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df4603e-31dd-4eea-b336-c8291db0a057 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:26.526251 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df4603e-31dd-4eea-b336-c8291db0a057 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:26.538394 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1337/2656] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:28:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 17:28:26.542786 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a228fd72-b439-4331-99a2-de64471a70c5 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:26.552018 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a228fd72-b439-4331-99a2-de64471a70c5 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:26.556996 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fee170f-58e9-47e6-af33-b1e7f8329d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:26.557588 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee170f-58e9-47e6-af33-b1e7f8329d51 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:26.557872 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee170f-58e9-47e6-af33-b1e7f8329d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:26.558145 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee170f-58e9-47e6-af33-b1e7f8329d51 admin admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:26.558761 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fee170f-58e9-47e6-af33-b1e7f8329d51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:26.560957 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fee170f-58e9-47e6-af33-b1e7f8329d51 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:26.661511 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1320/2657] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:28:26 2022] POST /identity/v3/projects => generated 381 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:28:26.666437 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df4603e-31dd-4eea-b336-c8291db0a057 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:26.676531 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df4603e-31dd-4eea-b336-c8291db0a057 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:26.727798 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:28:26.735153 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6087391-9884-436b-b9be-58c9486fb2cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:26.735760 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6087391-9884-436b-b9be-58c9486fb2cb admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:26.736113 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6087391-9884-436b-b9be-58c9486fb2cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:26.736337 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6087391-9884-436b-b9be-58c9486fb2cb admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:26.736963 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6087391-9884-436b-b9be-58c9486fb2cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:26.738918 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6087391-9884-436b-b9be-58c9486fb2cb admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:26.820324 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1338/2658] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:28:26 2022] POST /identity/v3/users => generated 291 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:28:26.824484 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fee170f-58e9-47e6-af33-b1e7f8329d51 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:26.835138 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fee170f-58e9-47e6-af33-b1e7f8329d51 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:26.839851 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb53358-7761-4d17-8030-0815c7a6ec8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:26.840452 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb53358-7761-4d17-8030-0815c7a6ec8c admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:26.840735 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb53358-7761-4d17-8030-0815c7a6ec8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:26.841010 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb53358-7761-4d17-8030-0815c7a6ec8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:26.841680 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb53358-7761-4d17-8030-0815c7a6ec8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:26.843336 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb53358-7761-4d17-8030-0815c7a6ec8c admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:26.853768 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1321/2659] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:28:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:28:26.857834 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6087391-9884-436b-b9be-58c9486fb2cb admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:26.869986 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6087391-9884-436b-b9be-58c9486fb2cb admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:26.875160 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c9358f-3b10-4061-81c9-2840aa14bb93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:26.875772 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9358f-3b10-4061-81c9-2840aa14bb93 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:26.876057 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9358f-3b10-4061-81c9-2840aa14bb93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:26.876331 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9358f-3b10-4061-81c9-2840aa14bb93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:26.876972 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c9358f-3b10-4061-81c9-2840aa14bb93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:26.878580 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c9358f-3b10-4061-81c9-2840aa14bb93 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:26.889660 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1339/2660] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:28:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:28:26.893861 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb53358-7761-4d17-8030-0815c7a6ec8c admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:26.904011 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb53358-7761-4d17-8030-0815c7a6ec8c admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:26.908612 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1851ee4a-7087-462c-be95-e85c97b362eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:26.909406 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1851ee4a-7087-462c-be95-e85c97b362eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:26.909683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1851ee4a-7087-462c-be95-e85c97b362eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:26.909956 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1851ee4a-7087-462c-be95-e85c97b362eb admin admin] PATH_INFO: `/v3/projects/ce647dfaf6e148208adc6322d16c88f5/users/860f6e9302fe46329aacc68e03d4ca46/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:26.924951 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1851ee4a-7087-462c-be95-e85c97b362eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce647dfaf6e148208adc6322d16c88f5, user_id=860f6e9302fe46329aacc68e03d4ca46, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:26.928345 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1851ee4a-7087-462c-be95-e85c97b362eb admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:27.008797 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1322/2661] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:28:26 2022] PUT /identity/v3/projects/ce647dfaf6e148208adc6322d16c88f5/users/860f6e9302fe46329aacc68e03d4ca46/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:28:27.019345 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a99732-cfa0-4a1f-a8de-93d05d5b3b8f None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:27.019636 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a99732-cfa0-4a1f-a8de-93d05d5b3b8f None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:27.019938 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a99732-cfa0-4a1f-a8de-93d05d5b3b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:27.049589 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-92a99732-cfa0-4a1f-a8de-93d05d5b3b8f None None] MFA Rules not processed for user `860f6e9302fe46329aacc68e03d4ca46`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:28:27.050812 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-92a99732-cfa0-4a1f-a8de-93d05d5b3b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:28:27.117458 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1340/2662] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:28:27 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:28:27.121990 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1851ee4a-7087-462c-be95-e85c97b362eb admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:27.183761 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1851ee4a-7087-462c-be95-e85c97b362eb admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:27.188704 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8844c7-8551-4b9a-8e65-95df54f71d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:27.189314 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8844c7-8551-4b9a-8e65-95df54f71d91 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:27.189598 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8844c7-8551-4b9a-8e65-95df54f71d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:27.189878 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8844c7-8551-4b9a-8e65-95df54f71d91 admin admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:27.190492 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8844c7-8551-4b9a-8e65-95df54f71d91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:27.192456 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8844c7-8551-4b9a-8e65-95df54f71d91 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:27.260471 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1323/2663] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:28:27 2022] POST /identity/v3/projects => generated 383 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:28:27.264619 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a99732-cfa0-4a1f-a8de-93d05d5b3b8f None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:27.276653 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a99732-cfa0-4a1f-a8de-93d05d5b3b8f None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:27.335738 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8652331-bb25-4e2b-b00d-6eb418d8d38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:27.336261 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8652331-bb25-4e2b-b00d-6eb418d8d38f admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:27.336547 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8652331-bb25-4e2b-b00d-6eb418d8d38f admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:27.336819 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8652331-bb25-4e2b-b00d-6eb418d8d38f admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:27.337424 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8652331-bb25-4e2b-b00d-6eb418d8d38f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:27.339436 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8652331-bb25-4e2b-b00d-6eb418d8d38f admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:27.444147 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1341/2664] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:28:27 2022] POST /identity/v3/users => generated 298 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:28:27.448306 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8844c7-8551-4b9a-8e65-95df54f71d91 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:27.453969 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:28:27.459397 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8844c7-8551-4b9a-8e65-95df54f71d91 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:27.464853 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6eb62e-89ab-4360-8356-c493c4b2bf8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:27.465457 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6eb62e-89ab-4360-8356-c493c4b2bf8e admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:27.465735 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6eb62e-89ab-4360-8356-c493c4b2bf8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:27.466007 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6eb62e-89ab-4360-8356-c493c4b2bf8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:27.466888 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6eb62e-89ab-4360-8356-c493c4b2bf8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:27.468513 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6eb62e-89ab-4360-8356-c493c4b2bf8e admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:27.480642 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1324/2665] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:28:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:28:27.484982 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8652331-bb25-4e2b-b00d-6eb418d8d38f admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:27.496284 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8652331-bb25-4e2b-b00d-6eb418d8d38f admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:27.501131 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263bd468-0141-4795-a8b0-ad6cea8f5950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:27.502233 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263bd468-0141-4795-a8b0-ad6cea8f5950 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:27.502233 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263bd468-0141-4795-a8b0-ad6cea8f5950 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:27.502437 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263bd468-0141-4795-a8b0-ad6cea8f5950 admin admin] PATH_INFO: `/v3/projects/9393565425594c1ab678c027febcc7d4/users/4a202ee8ae08459081716826deca2233/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:27.517582 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263bd468-0141-4795-a8b0-ad6cea8f5950 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9393565425594c1ab678c027febcc7d4, user_id=4a202ee8ae08459081716826deca2233, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:27.520415 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263bd468-0141-4795-a8b0-ad6cea8f5950 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:27.591792 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1342/2666] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:28:27 2022] PUT /identity/v3/projects/9393565425594c1ab678c027febcc7d4/users/4a202ee8ae08459081716826deca2233/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:28:27.595735 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6eb62e-89ab-4360-8356-c493c4b2bf8e admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:27.606432 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6eb62e-89ab-4360-8356-c493c4b2bf8e admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:27.661597 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bad48f-719a-4d80-8729-eae1ef309332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:27.663002 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bad48f-719a-4d80-8729-eae1ef309332 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:27.663288 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bad48f-719a-4d80-8729-eae1ef309332 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:27.663559 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bad48f-719a-4d80-8729-eae1ef309332 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:27.664182 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bad48f-719a-4d80-8729-eae1ef309332 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:27.665812 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bad48f-719a-4d80-8729-eae1ef309332 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:27.675286 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1325/2667] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:28:27 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 17:28:27.680153 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263bd468-0141-4795-a8b0-ad6cea8f5950 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:27.690531 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263bd468-0141-4795-a8b0-ad6cea8f5950 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:27.695160 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d45671-8126-4e09-8009-88eb46dde3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:27.695839 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d45671-8126-4e09-8009-88eb46dde3fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:27.696118 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d45671-8126-4e09-8009-88eb46dde3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:27.696391 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d45671-8126-4e09-8009-88eb46dde3fb admin admin] PATH_INFO: `/v3/projects/9393565425594c1ab678c027febcc7d4/users/4a202ee8ae08459081716826deca2233/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:27.698480 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d45671-8126-4e09-8009-88eb46dde3fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=9393565425594c1ab678c027febcc7d4, user_id=4a202ee8ae08459081716826deca2233, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:27.701139 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d45671-8126-4e09-8009-88eb46dde3fb admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:27.712203 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1343/2668] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:28:27 2022] PUT /identity/v3/projects/9393565425594c1ab678c027febcc7d4/users/4a202ee8ae08459081716826deca2233/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:28:27.723362 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e0dab-067d-4245-99a9-ba46d75a71c8 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:27.726751 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e0dab-067d-4245-99a9-ba46d75a71c8 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:27.726751 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e0dab-067d-4245-99a9-ba46d75a71c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:27.752301 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-7f4e0dab-067d-4245-99a9-ba46d75a71c8 None None] MFA Rules not processed for user `4a202ee8ae08459081716826deca2233`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:28:27.755628 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-7f4e0dab-067d-4245-99a9-ba46d75a71c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:28:27.822166 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1326/2669] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:28:27 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:28:27.832983 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d45671-8126-4e09-8009-88eb46dde3fb admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:27.835300 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-55d45671-8126-4e09-8009-88eb46dde3fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:28:27.854948 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:28:27.892735 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d45671-8126-4e09-8009-88eb46dde3fb admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:27.896928 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52d19f6-12be-4bac-9937-27a2c6c788cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:27.897610 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d19f6-12be-4bac-9937-27a2c6c788cb service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:27.897894 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d19f6-12be-4bac-9937-27a2c6c788cb service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:27.898171 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d19f6-12be-4bac-9937-27a2c6c788cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:27.907607 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52d19f6-12be-4bac-9937-27a2c6c788cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:27.909036 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52d19f6-12be-4bac-9937-27a2c6c788cb service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:27.919083 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1344/2670] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:28:27 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:28:28.024627 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4e0dab-067d-4245-99a9-ba46d75a71c8 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:28.034483 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4e0dab-067d-4245-99a9-ba46d75a71c8 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:28.038557 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e07fba-66a6-468c-936a-58b6d3b8cb0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:28.039183 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e07fba-66a6-468c-936a-58b6d3b8cb0c service cinder] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:28.039464 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e07fba-66a6-468c-936a-58b6d3b8cb0c service cinder] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:28.039737 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e07fba-66a6-468c-936a-58b6d3b8cb0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:28.096072 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e07fba-66a6-468c-936a-58b6d3b8cb0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:28.097523 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e07fba-66a6-468c-936a-58b6d3b8cb0c service cinder] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:28.106769 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1327/2671] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:28:28 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:28:31.141607 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52d19f6-12be-4bac-9937-27a2c6c788cb service cinder] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:31.204267 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52d19f6-12be-4bac-9937-27a2c6c788cb service cinder] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:31.208858 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6b1575-3e93-400e-8709-2091ac4b9c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:31.209477 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6b1575-3e93-400e-8709-2091ac4b9c4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:31.209754 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6b1575-3e93-400e-8709-2091ac4b9c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:31.210022 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6b1575-3e93-400e-8709-2091ac4b9c4e admin admin] PATH_INFO: `/v3/users/860f6e9302fe46329aacc68e03d4ca46` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:31.211286 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6b1575-3e93-400e-8709-2091ac4b9c4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=860f6e9302fe46329aacc68e03d4ca46)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:31.213178 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6b1575-3e93-400e-8709-2091ac4b9c4e admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:31.444637 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-fd6b1575-3e93-400e-8709-2091ac4b9c4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '860f6e9302fe46329aacc68e03d4ca46'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:31.510608 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-fd6b1575-3e93-400e-8709-2091ac4b9c4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '860f6e9302fe46329aacc68e03d4ca46'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:31.511376 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-fd6b1575-3e93-400e-8709-2091ac4b9c4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '860f6e9302fe46329aacc68e03d4ca46'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:31.511986 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-fd6b1575-3e93-400e-8709-2091ac4b9c4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '860f6e9302fe46329aacc68e03d4ca46'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:31.519042 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-fd6b1575-3e93-400e-8709-2091ac4b9c4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '860f6e9302fe46329aacc68e03d4ca46'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:31.535592 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1345/2672] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:28:31 2022] DELETE /identity/v3/users/860f6e9302fe46329aacc68e03d4ca46 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:28:31.540253 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a6799-e57c-4149-8f6e-736399d26c81 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:31.540734 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a6799-e57c-4149-8f6e-736399d26c81 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:31.540829 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a6799-e57c-4149-8f6e-736399d26c81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:31.562286 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-163a6799-e57c-4149-8f6e-736399d26c81 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:28:31.563594 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-163a6799-e57c-4149-8f6e-736399d26c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:28:31.568612 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=100 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:28:31.621077 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1328/2673] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:28:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:28:31.630777 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6b1575-3e93-400e-8709-2091ac4b9c4e admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:31.632207 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-fd6b1575-3e93-400e-8709-2091ac4b9c4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:28:31.674313 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:28:31.690135 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6b1575-3e93-400e-8709-2091ac4b9c4e admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:31.693885 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1362c89-d23e-4fb7-ba2f-c75da7166dca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:31.694490 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1362c89-d23e-4fb7-ba2f-c75da7166dca service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:31.694822 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1362c89-d23e-4fb7-ba2f-c75da7166dca service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:31.695088 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1362c89-d23e-4fb7-ba2f-c75da7166dca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:31.704827 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1362c89-d23e-4fb7-ba2f-c75da7166dca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:31.706566 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1362c89-d23e-4fb7-ba2f-c75da7166dca service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:31.717252 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1346/2674] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:28:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:28:32.130256 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163a6799-e57c-4149-8f6e-736399d26c81 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:32.131743 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-163a6799-e57c-4149-8f6e-736399d26c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:28:32.144983 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163a6799-e57c-4149-8f6e-736399d26c81 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:32.150830 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fa0e6d-be83-4078-bdfa-fd234a77f05c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:32.151489 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa0e6d-be83-4078-bdfa-fd234a77f05c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:32.151771 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa0e6d-be83-4078-bdfa-fd234a77f05c admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:32.152043 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa0e6d-be83-4078-bdfa-fd234a77f05c admin admin] PATH_INFO: `/v3/projects/ce647dfaf6e148208adc6322d16c88f5` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:32.153144 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fa0e6d-be83-4078-bdfa-fd234a77f05c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce647dfaf6e148208adc6322d16c88f5)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:32.155074 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fa0e6d-be83-4078-bdfa-fd234a77f05c admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:32.254743 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-a0fa0e6d-be83-4078-bdfa-fd234a77f05c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce647dfaf6e148208adc6322d16c88f5', 'request_id': 'req-a0fa0e6d-be83-4078-bdfa-fd234a77f05c'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:32.323033 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-a0fa0e6d-be83-4078-bdfa-fd234a77f05c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce647dfaf6e148208adc6322d16c88f5', 'request_id': 'req-a0fa0e6d-be83-4078-bdfa-fd234a77f05c'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:32.329212 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-a0fa0e6d-be83-4078-bdfa-fd234a77f05c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce647dfaf6e148208adc6322d16c88f5', 'request_id': 'req-a0fa0e6d-be83-4078-bdfa-fd234a77f05c'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:32.341381 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-a0fa0e6d-be83-4078-bdfa-fd234a77f05c admin admin] The token cache is being invalidate because project ce647dfaf6e148208adc6322d16c88f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81650) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:28:32.341669 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-a0fa0e6d-be83-4078-bdfa-fd234a77f05c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:32.342299 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-a0fa0e6d-be83-4078-bdfa-fd234a77f05c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:32.343811 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1329/2675] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:28:32 2022] DELETE /identity/v3/projects/ce647dfaf6e148208adc6322d16c88f5 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:28:32.347772 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1362c89-d23e-4fb7-ba2f-c75da7166dca service neutron] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:32.358325 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1362c89-d23e-4fb7-ba2f-c75da7166dca service neutron] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:32.415139 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd04697d-77a5-4955-a37c-6d6bb94ec9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:32.415762 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd04697d-77a5-4955-a37c-6d6bb94ec9db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:32.416037 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd04697d-77a5-4955-a37c-6d6bb94ec9db admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:32.416303 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd04697d-77a5-4955-a37c-6d6bb94ec9db admin admin] PATH_INFO: `/v3/users/4a202ee8ae08459081716826deca2233` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:32.417401 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd04697d-77a5-4955-a37c-6d6bb94ec9db admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a202ee8ae08459081716826deca2233)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:32.419367 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd04697d-77a5-4955-a37c-6d6bb94ec9db admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:32.576742 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-cd04697d-77a5-4955-a37c-6d6bb94ec9db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a202ee8ae08459081716826deca2233'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:32.636098 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-cd04697d-77a5-4955-a37c-6d6bb94ec9db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a202ee8ae08459081716826deca2233'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:32.636727 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-cd04697d-77a5-4955-a37c-6d6bb94ec9db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a202ee8ae08459081716826deca2233'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:32.637383 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-cd04697d-77a5-4955-a37c-6d6bb94ec9db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a202ee8ae08459081716826deca2233'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:32.644515 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-cd04697d-77a5-4955-a37c-6d6bb94ec9db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a202ee8ae08459081716826deca2233'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:32.662535 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1347/2676] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:28:32 2022] DELETE /identity/v3/users/4a202ee8ae08459081716826deca2233 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:28:33.062798 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fa0e6d-be83-4078-bdfa-fd234a77f05c admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:33.064256 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-a0fa0e6d-be83-4078-bdfa-fd234a77f05c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:28:33.081419 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fa0e6d-be83-4078-bdfa-fd234a77f05c admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:33.140053 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0aa06b-9122-423e-9df2-eed02e1e3db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:33.140695 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0aa06b-9122-423e-9df2-eed02e1e3db2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:33.140966 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0aa06b-9122-423e-9df2-eed02e1e3db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:33.141243 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0aa06b-9122-423e-9df2-eed02e1e3db2 admin admin] PATH_INFO: `/v3/projects/9393565425594c1ab678c027febcc7d4` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:33.142352 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0aa06b-9122-423e-9df2-eed02e1e3db2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9393565425594c1ab678c027febcc7d4)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:33.144307 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0aa06b-9122-423e-9df2-eed02e1e3db2 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:33.223236 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:28:33.232559 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-8a0aa06b-9122-423e-9df2-eed02e1e3db2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9393565425594c1ab678c027febcc7d4', 'request_id': 'req-8a0aa06b-9122-423e-9df2-eed02e1e3db2'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:33.300001 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-8a0aa06b-9122-423e-9df2-eed02e1e3db2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9393565425594c1ab678c027febcc7d4', 'request_id': 'req-8a0aa06b-9122-423e-9df2-eed02e1e3db2'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:33.305906 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-8a0aa06b-9122-423e-9df2-eed02e1e3db2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9393565425594c1ab678c027febcc7d4', 'request_id': 'req-8a0aa06b-9122-423e-9df2-eed02e1e3db2'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:33.317778 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-8a0aa06b-9122-423e-9df2-eed02e1e3db2 admin admin] The token cache is being invalidate because project 9393565425594c1ab678c027febcc7d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81650) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:28:33.323371 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-8a0aa06b-9122-423e-9df2-eed02e1e3db2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:33.323371 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-8a0aa06b-9122-423e-9df2-eed02e1e3db2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:33.323371 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1330/2677] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:28:33 2022] DELETE /identity/v3/projects/9393565425594c1ab678c027febcc7d4 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:28:33.325590 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061170d9-f01b-47d9-97ab-cc49bc47ffed None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:33.325898 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061170d9-f01b-47d9-97ab-cc49bc47ffed None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:33.326176 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061170d9-f01b-47d9-97ab-cc49bc47ffed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:33.347164 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-061170d9-f01b-47d9-97ab-cc49bc47ffed None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:28:33.348291 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-061170d9-f01b-47d9-97ab-cc49bc47ffed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:28:33.402711 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:28:33.411084 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1348/2678] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:28:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:28:33.473745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06898ff1-03cc-4568-ab6e-71a5621817a5 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:33.474032 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06898ff1-03cc-4568-ab6e-71a5621817a5 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:33.474311 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06898ff1-03cc-4568-ab6e-71a5621817a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:33.495066 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-06898ff1-03cc-4568-ab6e-71a5621817a5 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:28:33.496267 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-06898ff1-03cc-4568-ab6e-71a5621817a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:28:33.504085 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1331/2679] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:28:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:28:33.508335 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061170d9-f01b-47d9-97ab-cc49bc47ffed None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:33.517979 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061170d9-f01b-47d9-97ab-cc49bc47ffed None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:33.522679 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8123dda9-9de0-4195-8bef-6c067c5f96c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:33.523361 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8123dda9-9de0-4195-8bef-6c067c5f96c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:33.523786 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8123dda9-9de0-4195-8bef-6c067c5f96c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:33.524062 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8123dda9-9de0-4195-8bef-6c067c5f96c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:33.524709 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8123dda9-9de0-4195-8bef-6c067c5f96c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:28:33.525015 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8123dda9-9de0-4195-8bef-6c067c5f96c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:33.526674 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8123dda9-9de0-4195-8bef-6c067c5f96c8 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:33.538370 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1349/2680] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:28:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 17:28:33.543298 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06898ff1-03cc-4568-ab6e-71a5621817a5 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:33.552581 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06898ff1-03cc-4568-ab6e-71a5621817a5 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:33.558061 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f195dca-564c-44e0-9015-e7b297f66ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:33.558927 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f195dca-564c-44e0-9015-e7b297f66ec3 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:33.559217 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f195dca-564c-44e0-9015-e7b297f66ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:33.559493 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f195dca-564c-44e0-9015-e7b297f66ec3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:33.560121 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f195dca-564c-44e0-9015-e7b297f66ec3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:33.562039 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f195dca-564c-44e0-9015-e7b297f66ec3 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:33.644576 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1332/2681] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:28:33 2022] POST /identity/v3/projects => generated 383 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:28:33.648418 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8123dda9-9de0-4195-8bef-6c067c5f96c8 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:33.658932 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8123dda9-9de0-4195-8bef-6c067c5f96c8 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:33.716007 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ce9803-44b2-4eda-a6af-439a8ab1b138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:33.716663 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ce9803-44b2-4eda-a6af-439a8ab1b138 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:33.716950 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ce9803-44b2-4eda-a6af-439a8ab1b138 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:33.717226 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ce9803-44b2-4eda-a6af-439a8ab1b138 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:33.717837 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ce9803-44b2-4eda-a6af-439a8ab1b138 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:33.719786 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ce9803-44b2-4eda-a6af-439a8ab1b138 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:33.803038 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1350/2682] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:28:33 2022] POST /identity/v3/users => generated 292 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:28:33.807184 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f195dca-564c-44e0-9015-e7b297f66ec3 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:33.817891 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f195dca-564c-44e0-9015-e7b297f66ec3 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:33.822668 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bd2a4e-5914-4afd-b617-1a84f72267d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:33.823301 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd2a4e-5914-4afd-b617-1a84f72267d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:33.823584 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd2a4e-5914-4afd-b617-1a84f72267d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:33.823859 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd2a4e-5914-4afd-b617-1a84f72267d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:33.824569 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bd2a4e-5914-4afd-b617-1a84f72267d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:33.826235 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bd2a4e-5914-4afd-b617-1a84f72267d7 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:33.836687 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1333/2683] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:28:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:28:33.840722 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ce9803-44b2-4eda-a6af-439a8ab1b138 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:33.850716 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ce9803-44b2-4eda-a6af-439a8ab1b138 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:33.856197 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaf297b-4e32-4833-94b9-7670cfb9771c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:33.856785 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf297b-4e32-4833-94b9-7670cfb9771c admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:33.857063 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf297b-4e32-4833-94b9-7670cfb9771c admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:33.857332 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf297b-4e32-4833-94b9-7670cfb9771c admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:33.857953 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaf297b-4e32-4833-94b9-7670cfb9771c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:33.859688 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaf297b-4e32-4833-94b9-7670cfb9771c admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:33.870592 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1351/2684] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:28:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:28:33.874576 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bd2a4e-5914-4afd-b617-1a84f72267d7 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:33.884559 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bd2a4e-5914-4afd-b617-1a84f72267d7 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:33.889242 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a050995-2b84-464c-af4c-7e45b99c3860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:33.889881 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a050995-2b84-464c-af4c-7e45b99c3860 admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:33.890165 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a050995-2b84-464c-af4c-7e45b99c3860 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:33.890443 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a050995-2b84-464c-af4c-7e45b99c3860 admin admin] PATH_INFO: `/v3/projects/f3625eec68b64270989e39555b317107/users/bd349ba1bc7a46848fd01fb43e5a4b48/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:33.905203 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a050995-2b84-464c-af4c-7e45b99c3860 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3625eec68b64270989e39555b317107, user_id=bd349ba1bc7a46848fd01fb43e5a4b48, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:33.907843 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a050995-2b84-464c-af4c-7e45b99c3860 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:33.963071 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1334/2685] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:28:33 2022] PUT /identity/v3/projects/f3625eec68b64270989e39555b317107/users/bd349ba1bc7a46848fd01fb43e5a4b48/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:28:33.973266 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747791ec-b35d-4509-bf60-ed5494fc32d5 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:33.973552 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747791ec-b35d-4509-bf60-ed5494fc32d5 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:33.973824 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747791ec-b35d-4509-bf60-ed5494fc32d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:34.001686 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-747791ec-b35d-4509-bf60-ed5494fc32d5 None None] MFA Rules not processed for user `bd349ba1bc7a46848fd01fb43e5a4b48`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:28:34.002975 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-747791ec-b35d-4509-bf60-ed5494fc32d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:28:34.065802 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1352/2686] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:28:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:28:34.070294 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a050995-2b84-464c-af4c-7e45b99c3860 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:34.132319 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a050995-2b84-464c-af4c-7e45b99c3860 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:34.137779 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9808de5-ba12-4547-9d4f-78f9311ff3e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:34.138377 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9808de5-ba12-4547-9d4f-78f9311ff3e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:34.138661 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9808de5-ba12-4547-9d4f-78f9311ff3e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:34.138957 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9808de5-ba12-4547-9d4f-78f9311ff3e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:34.139585 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9808de5-ba12-4547-9d4f-78f9311ff3e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:34.141483 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9808de5-ba12-4547-9d4f-78f9311ff3e5 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:34.233057 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1335/2687] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:28:34 2022] POST /identity/v3/projects => generated 383 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:28:34.237061 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747791ec-b35d-4509-bf60-ed5494fc32d5 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:34.248263 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747791ec-b35d-4509-bf60-ed5494fc32d5 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:34.297996 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:28:34.305069 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f478dfd6-75b4-4219-8693-c105ee771d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:34.305777 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f478dfd6-75b4-4219-8693-c105ee771d2a admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:34.306059 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f478dfd6-75b4-4219-8693-c105ee771d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:34.306329 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f478dfd6-75b4-4219-8693-c105ee771d2a admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:34.306970 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f478dfd6-75b4-4219-8693-c105ee771d2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:34.309194 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f478dfd6-75b4-4219-8693-c105ee771d2a admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:34.396875 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1353/2688] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:28:34 2022] POST /identity/v3/users => generated 298 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 17:28:34.400932 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9808de5-ba12-4547-9d4f-78f9311ff3e5 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:34.411361 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9808de5-ba12-4547-9d4f-78f9311ff3e5 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:34.416396 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f6e1b7-e5b4-4379-aa90-47f4c2ddf861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:34.416998 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f6e1b7-e5b4-4379-aa90-47f4c2ddf861 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:34.417280 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f6e1b7-e5b4-4379-aa90-47f4c2ddf861 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:34.417553 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f6e1b7-e5b4-4379-aa90-47f4c2ddf861 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:34.418281 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f6e1b7-e5b4-4379-aa90-47f4c2ddf861 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:34.419924 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f6e1b7-e5b4-4379-aa90-47f4c2ddf861 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:34.430982 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1336/2689] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:28:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:28:34.435293 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f478dfd6-75b4-4219-8693-c105ee771d2a admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:34.445079 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f478dfd6-75b4-4219-8693-c105ee771d2a admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:34.450471 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60513280-65e0-4308-a3d5-1ebe2514431e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:34.451141 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60513280-65e0-4308-a3d5-1ebe2514431e admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:34.451418 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60513280-65e0-4308-a3d5-1ebe2514431e admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:34.451685 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60513280-65e0-4308-a3d5-1ebe2514431e admin admin] PATH_INFO: `/v3/projects/6d7e319d44ff405b984bee5e1c960da5/users/feea1006dc4743cbb1646a902ef378d1/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:34.466181 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60513280-65e0-4308-a3d5-1ebe2514431e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d7e319d44ff405b984bee5e1c960da5, user_id=feea1006dc4743cbb1646a902ef378d1, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:34.468776 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60513280-65e0-4308-a3d5-1ebe2514431e admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:34.534411 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1354/2690] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:28:34 2022] PUT /identity/v3/projects/6d7e319d44ff405b984bee5e1c960da5/users/feea1006dc4743cbb1646a902ef378d1/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:28:34.538311 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f6e1b7-e5b4-4379-aa90-47f4c2ddf861 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:34.548558 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f6e1b7-e5b4-4379-aa90-47f4c2ddf861 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:34.581719 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:28:34.605579 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5da992-1ad7-478a-a8db-001c38a8ff24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:34.606189 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5da992-1ad7-478a-a8db-001c38a8ff24 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:34.606477 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5da992-1ad7-478a-a8db-001c38a8ff24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:34.606756 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5da992-1ad7-478a-a8db-001c38a8ff24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:34.607538 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5da992-1ad7-478a-a8db-001c38a8ff24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:34.609106 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5da992-1ad7-478a-a8db-001c38a8ff24 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:34.619740 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1337/2691] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:28:34 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:28:34.623659 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60513280-65e0-4308-a3d5-1ebe2514431e admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:34.634659 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60513280-65e0-4308-a3d5-1ebe2514431e admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:34.639377 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3385a57e-0620-4c12-8911-f7755dcc62c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:34.640018 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3385a57e-0620-4c12-8911-f7755dcc62c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:34.640295 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3385a57e-0620-4c12-8911-f7755dcc62c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:34.640565 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3385a57e-0620-4c12-8911-f7755dcc62c7 admin admin] PATH_INFO: `/v3/projects/6d7e319d44ff405b984bee5e1c960da5/users/feea1006dc4743cbb1646a902ef378d1/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:34.642608 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3385a57e-0620-4c12-8911-f7755dcc62c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d7e319d44ff405b984bee5e1c960da5, user_id=feea1006dc4743cbb1646a902ef378d1, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:34.645279 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3385a57e-0620-4c12-8911-f7755dcc62c7 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:34.655731 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1355/2692] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:28:34 2022] PUT /identity/v3/projects/6d7e319d44ff405b984bee5e1c960da5/users/feea1006dc4743cbb1646a902ef378d1/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:28:34.667163 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1274f36-5de2-463a-8b58-78725d222bf7 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:34.667480 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1274f36-5de2-463a-8b58-78725d222bf7 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:34.667762 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1274f36-5de2-463a-8b58-78725d222bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:34.697902 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-f1274f36-5de2-463a-8b58-78725d222bf7 None None] MFA Rules not processed for user `feea1006dc4743cbb1646a902ef378d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:28:34.699212 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-f1274f36-5de2-463a-8b58-78725d222bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:28:34.769456 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1338/2693] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:28:34 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:28:34.780321 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3385a57e-0620-4c12-8911-f7755dcc62c7 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:34.782025 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-3385a57e-0620-4c12-8911-f7755dcc62c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:28:34.839100 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3385a57e-0620-4c12-8911-f7755dcc62c7 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:34.842768 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d83e18-4d69-4281-b186-74d024cb4bf9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:34.843403 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d83e18-4d69-4281-b186-74d024cb4bf9 service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:34.843686 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d83e18-4d69-4281-b186-74d024cb4bf9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:34.843959 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d83e18-4d69-4281-b186-74d024cb4bf9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:34.899198 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d83e18-4d69-4281-b186-74d024cb4bf9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:34.900616 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d83e18-4d69-4281-b186-74d024cb4bf9 service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:34.909290 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1356/2694] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:28:34 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:28:40.066136 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1274f36-5de2-463a-8b58-78725d222bf7 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:40.067617 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-f1274f36-5de2-463a-8b58-78725d222bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:28:40.080161 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1274f36-5de2-463a-8b58-78725d222bf7 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:40.083841 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a010122-9176-4161-b9a2-56ecb7261f99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:40.084432 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a010122-9176-4161-b9a2-56ecb7261f99 service cinder] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:40.084708 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a010122-9176-4161-b9a2-56ecb7261f99 service cinder] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:40.084975 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a010122-9176-4161-b9a2-56ecb7261f99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:40.094732 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a010122-9176-4161-b9a2-56ecb7261f99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:40.096192 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a010122-9176-4161-b9a2-56ecb7261f99 service cinder] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:40.105640 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1339/2695] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:28:40 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:28:44.905337 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=67,keystone:INSERT=5 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:28:45.754972 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d83e18-4d69-4281-b186-74d024cb4bf9 service cinder] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:45.826275 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d83e18-4d69-4281-b186-74d024cb4bf9 service cinder] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:45.831091 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f2733-c8e3-4453-92b4-86af39840297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:45.831753 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f2733-c8e3-4453-92b4-86af39840297 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:45.832035 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f2733-c8e3-4453-92b4-86af39840297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:45.832311 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f2733-c8e3-4453-92b4-86af39840297 admin admin] PATH_INFO: `/v3/users/bd349ba1bc7a46848fd01fb43e5a4b48` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:45.833445 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63f2733-c8e3-4453-92b4-86af39840297 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd349ba1bc7a46848fd01fb43e5a4b48)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:45.835442 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63f2733-c8e3-4453-92b4-86af39840297 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:46.021534 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e63f2733-c8e3-4453-92b4-86af39840297 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd349ba1bc7a46848fd01fb43e5a4b48'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:46.111385 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e63f2733-c8e3-4453-92b4-86af39840297 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd349ba1bc7a46848fd01fb43e5a4b48'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:46.112002 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e63f2733-c8e3-4453-92b4-86af39840297 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd349ba1bc7a46848fd01fb43e5a4b48'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:46.112609 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e63f2733-c8e3-4453-92b4-86af39840297 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd349ba1bc7a46848fd01fb43e5a4b48'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:46.121050 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e63f2733-c8e3-4453-92b4-86af39840297 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd349ba1bc7a46848fd01fb43e5a4b48'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:46.138612 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1357/2696] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:28:45 2022] DELETE /identity/v3/users/bd349ba1bc7a46848fd01fb43e5a4b48 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:28:46.143345 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaaad65-be7d-4c98-b6d7-6d8c68032b0e None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:46.143840 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaaad65-be7d-4c98-b6d7-6d8c68032b0e None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:46.143965 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaaad65-be7d-4c98-b6d7-6d8c68032b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:46.163308 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-cbaaad65-be7d-4c98-b6d7-6d8c68032b0e None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:28:46.164517 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-cbaaad65-be7d-4c98-b6d7-6d8c68032b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:28:46.223770 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1340/2697] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:28:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:28:46.234271 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f2733-c8e3-4453-92b4-86af39840297 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:46.235747 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-e63f2733-c8e3-4453-92b4-86af39840297 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:28:46.293870 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f2733-c8e3-4453-92b4-86af39840297 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:46.297489 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5915911-f1b7-4a43-b51f-c494804b0ece service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:46.298123 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5915911-f1b7-4a43-b51f-c494804b0ece service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:46.298405 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5915911-f1b7-4a43-b51f-c494804b0ece service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:46.298706 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5915911-f1b7-4a43-b51f-c494804b0ece service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:46.309016 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5915911-f1b7-4a43-b51f-c494804b0ece service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:46.310498 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5915911-f1b7-4a43-b51f-c494804b0ece service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:46.319799 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1358/2698] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:28:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:28:46.769785 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaaad65-be7d-4c98-b6d7-6d8c68032b0e None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:46.771262 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-cbaaad65-be7d-4c98-b6d7-6d8c68032b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:28:46.784239 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaaad65-be7d-4c98-b6d7-6d8c68032b0e None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:46.790743 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8955743f-78f1-46a0-9376-e365220ace5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:46.791707 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955743f-78f1-46a0-9376-e365220ace5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:46.792017 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955743f-78f1-46a0-9376-e365220ace5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:46.792366 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955743f-78f1-46a0-9376-e365220ace5e admin admin] PATH_INFO: `/v3/projects/f3625eec68b64270989e39555b317107` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:46.793404 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8955743f-78f1-46a0-9376-e365220ace5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3625eec68b64270989e39555b317107)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:46.795347 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8955743f-78f1-46a0-9376-e365220ace5e admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:46.898971 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:28:46.906329 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-8955743f-78f1-46a0-9376-e365220ace5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3625eec68b64270989e39555b317107', 'request_id': 'req-8955743f-78f1-46a0-9376-e365220ace5e'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:46.950849 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-8955743f-78f1-46a0-9376-e365220ace5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3625eec68b64270989e39555b317107', 'request_id': 'req-8955743f-78f1-46a0-9376-e365220ace5e'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:46.956527 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-8955743f-78f1-46a0-9376-e365220ace5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3625eec68b64270989e39555b317107', 'request_id': 'req-8955743f-78f1-46a0-9376-e365220ace5e'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:46.968862 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-8955743f-78f1-46a0-9376-e365220ace5e admin admin] The token cache is being invalidate because project f3625eec68b64270989e39555b317107 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81650) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:28:46.969150 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-8955743f-78f1-46a0-9376-e365220ace5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:46.969791 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-8955743f-78f1-46a0-9376-e365220ace5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:46.971298 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1341/2699] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:28:46 2022] DELETE /identity/v3/projects/f3625eec68b64270989e39555b317107 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:28:46.975446 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5915911-f1b7-4a43-b51f-c494804b0ece service neutron] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:46.985184 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5915911-f1b7-4a43-b51f-c494804b0ece service neutron] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:47.002705 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:28:47.041984 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b5ab2c-9a3e-405d-ba7a-a9c97c2ec613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:47.042627 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5ab2c-9a3e-405d-ba7a-a9c97c2ec613 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:47.042875 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5ab2c-9a3e-405d-ba7a-a9c97c2ec613 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:47.043188 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5ab2c-9a3e-405d-ba7a-a9c97c2ec613 admin admin] PATH_INFO: `/v3/users/feea1006dc4743cbb1646a902ef378d1` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:47.044382 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b5ab2c-9a3e-405d-ba7a-a9c97c2ec613 admin admin] RBAC: Authorizing `identity:delete_user(user_id=feea1006dc4743cbb1646a902ef378d1)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:47.046354 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b5ab2c-9a3e-405d-ba7a-a9c97c2ec613 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:47.204912 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-d6b5ab2c-9a3e-405d-ba7a-a9c97c2ec613 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'feea1006dc4743cbb1646a902ef378d1'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:47.253667 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-d6b5ab2c-9a3e-405d-ba7a-a9c97c2ec613 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'feea1006dc4743cbb1646a902ef378d1'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:47.254304 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-d6b5ab2c-9a3e-405d-ba7a-a9c97c2ec613 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'feea1006dc4743cbb1646a902ef378d1'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:47.255000 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-d6b5ab2c-9a3e-405d-ba7a-a9c97c2ec613 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'feea1006dc4743cbb1646a902ef378d1'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:47.262464 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-d6b5ab2c-9a3e-405d-ba7a-a9c97c2ec613 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'feea1006dc4743cbb1646a902ef378d1'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:47.279943 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1359/2700] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:28:46 2022] DELETE /identity/v3/users/feea1006dc4743cbb1646a902ef378d1 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:28:47.622153 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8955743f-78f1-46a0-9376-e365220ace5e admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:47.623766 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-8955743f-78f1-46a0-9376-e365220ace5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:28:47.641536 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8955743f-78f1-46a0-9376-e365220ace5e admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:47.700238 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcad134-b98c-4848-8890-dbcc826fe0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:47.700868 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcad134-b98c-4848-8890-dbcc826fe0da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:47.701152 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcad134-b98c-4848-8890-dbcc826fe0da admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:47.701456 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcad134-b98c-4848-8890-dbcc826fe0da admin admin] PATH_INFO: `/v3/projects/6d7e319d44ff405b984bee5e1c960da5` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:47.702562 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcad134-b98c-4848-8890-dbcc826fe0da admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d7e319d44ff405b984bee5e1c960da5)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:47.704509 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcad134-b98c-4848-8890-dbcc826fe0da admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:47.792647 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-0bcad134-b98c-4848-8890-dbcc826fe0da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d7e319d44ff405b984bee5e1c960da5', 'request_id': 'req-0bcad134-b98c-4848-8890-dbcc826fe0da'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:47.854628 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-0bcad134-b98c-4848-8890-dbcc826fe0da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d7e319d44ff405b984bee5e1c960da5', 'request_id': 'req-0bcad134-b98c-4848-8890-dbcc826fe0da'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:47.860742 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-0bcad134-b98c-4848-8890-dbcc826fe0da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d7e319d44ff405b984bee5e1c960da5', 'request_id': 'req-0bcad134-b98c-4848-8890-dbcc826fe0da'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:47.872613 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-0bcad134-b98c-4848-8890-dbcc826fe0da admin admin] The token cache is being invalidate because project 6d7e319d44ff405b984bee5e1c960da5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81650) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:28:47.872773 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-0bcad134-b98c-4848-8890-dbcc826fe0da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:47.873318 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-0bcad134-b98c-4848-8890-dbcc826fe0da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:28:47.874794 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1342/2701] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:28:47 2022] DELETE /identity/v3/projects/6d7e319d44ff405b984bee5e1c960da5 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:28:47.880899 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221f5fb0-c169-4160-ac5b-5d8a6dbc4272 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:47.881226 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221f5fb0-c169-4160-ac5b-5d8a6dbc4272 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:47.881503 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221f5fb0-c169-4160-ac5b-5d8a6dbc4272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:47.903559 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-221f5fb0-c169-4160-ac5b-5d8a6dbc4272 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:28:47.904820 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-221f5fb0-c169-4160-ac5b-5d8a6dbc4272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:28:47.963959 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1360/2702] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:28:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:28:47.972185 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463cc3c0-afc4-402c-b49a-c659ca1a655f None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:47.972481 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463cc3c0-afc4-402c-b49a-c659ca1a655f None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:47.972768 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463cc3c0-afc4-402c-b49a-c659ca1a655f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:47.992883 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-463cc3c0-afc4-402c-b49a-c659ca1a655f None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:28:47.994306 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-463cc3c0-afc4-402c-b49a-c659ca1a655f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:28:48.002207 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1343/2703] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:28:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:28:48.006333 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221f5fb0-c169-4160-ac5b-5d8a6dbc4272 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:48.016020 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221f5fb0-c169-4160-ac5b-5d8a6dbc4272 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:48.020552 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8246ee-572f-4ff9-b788-e60292f2d9a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:48.021174 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8246ee-572f-4ff9-b788-e60292f2d9a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:48.021448 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8246ee-572f-4ff9-b788-e60292f2d9a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:48.021741 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8246ee-572f-4ff9-b788-e60292f2d9a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:48.022381 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8246ee-572f-4ff9-b788-e60292f2d9a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:28:48.022676 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8246ee-572f-4ff9-b788-e60292f2d9a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:48.024298 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8246ee-572f-4ff9-b788-e60292f2d9a6 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:48.036396 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1361/2704] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:28:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 17:28:48.041475 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463cc3c0-afc4-402c-b49a-c659ca1a655f None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:48.050659 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463cc3c0-afc4-402c-b49a-c659ca1a655f None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:48.055289 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9d9fed-7628-4767-9c14-602544725cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:48.055886 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d9fed-7628-4767-9c14-602544725cd9 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:48.056219 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d9fed-7628-4767-9c14-602544725cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:48.056493 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d9fed-7628-4767-9c14-602544725cd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:48.057105 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9d9fed-7628-4767-9c14-602544725cd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:48.059309 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9d9fed-7628-4767-9c14-602544725cd9 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:48.219639 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1344/2705] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:28:48 2022] POST /identity/v3/projects => generated 375 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:28:48.223795 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8246ee-572f-4ff9-b788-e60292f2d9a6 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:48.234601 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8246ee-572f-4ff9-b788-e60292f2d9a6 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:48.260896 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:28:48.291921 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835b261e-b37a-4f90-b1cc-72686f503a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:48.292524 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b261e-b37a-4f90-b1cc-72686f503a88 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:48.292812 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b261e-b37a-4f90-b1cc-72686f503a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:48.293082 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b261e-b37a-4f90-b1cc-72686f503a88 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:48.293692 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835b261e-b37a-4f90-b1cc-72686f503a88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:48.295628 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835b261e-b37a-4f90-b1cc-72686f503a88 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:48.414261 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1362/2706] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:28:48 2022] POST /identity/v3/users => generated 288 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:28:48.418847 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9d9fed-7628-4767-9c14-602544725cd9 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:48.429327 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9d9fed-7628-4767-9c14-602544725cd9 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:48.434331 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3a60e6-37e1-4f2e-8c68-2854af9ab13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:48.434970 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a60e6-37e1-4f2e-8c68-2854af9ab13a admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:48.435256 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a60e6-37e1-4f2e-8c68-2854af9ab13a admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:48.435530 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a60e6-37e1-4f2e-8c68-2854af9ab13a admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:48.436225 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3a60e6-37e1-4f2e-8c68-2854af9ab13a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:48.437837 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3a60e6-37e1-4f2e-8c68-2854af9ab13a admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:48.448238 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1345/2707] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:28:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:28:48.452234 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835b261e-b37a-4f90-b1cc-72686f503a88 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:48.461978 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835b261e-b37a-4f90-b1cc-72686f503a88 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:48.467374 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a80321-a978-45a4-aad6-5d1ddfc209dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:48.467973 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a80321-a978-45a4-aad6-5d1ddfc209dd admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:48.468256 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a80321-a978-45a4-aad6-5d1ddfc209dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:48.468582 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a80321-a978-45a4-aad6-5d1ddfc209dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:48.469168 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a80321-a978-45a4-aad6-5d1ddfc209dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:48.470754 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a80321-a978-45a4-aad6-5d1ddfc209dd admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:48.480769 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1363/2708] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:28:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 17:28:48.484737 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3a60e6-37e1-4f2e-8c68-2854af9ab13a admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:48.494439 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3a60e6-37e1-4f2e-8c68-2854af9ab13a admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:48.499020 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d52398-4593-46a6-9a8e-3935e09ece8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:48.499665 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d52398-4593-46a6-9a8e-3935e09ece8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:48.499947 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d52398-4593-46a6-9a8e-3935e09ece8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:48.500221 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d52398-4593-46a6-9a8e-3935e09ece8f admin admin] PATH_INFO: `/v3/projects/40133fcf86b84853868830b693a6b9e2/users/10da2e2e93014fef9053f8870005a717/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:48.514761 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d52398-4593-46a6-9a8e-3935e09ece8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=40133fcf86b84853868830b693a6b9e2, user_id=10da2e2e93014fef9053f8870005a717, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:48.517439 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d52398-4593-46a6-9a8e-3935e09ece8f admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:48.519615 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:28:48.585170 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1346/2709] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:28:48 2022] PUT /identity/v3/projects/40133fcf86b84853868830b693a6b9e2/users/10da2e2e93014fef9053f8870005a717/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:28:48.597292 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9469538f-adb1-4e5c-9933-751a1d4257f4 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:48.597586 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9469538f-adb1-4e5c-9933-751a1d4257f4 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:48.597865 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9469538f-adb1-4e5c-9933-751a1d4257f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:48.625585 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-9469538f-adb1-4e5c-9933-751a1d4257f4 None None] MFA Rules not processed for user `10da2e2e93014fef9053f8870005a717`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:28:48.626800 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-9469538f-adb1-4e5c-9933-751a1d4257f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:28:48.690492 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1364/2710] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:28:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:28:48.702655 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d52398-4593-46a6-9a8e-3935e09ece8f admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:28:48.704382 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-99d52398-4593-46a6-9a8e-3935e09ece8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:28:48.760971 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d52398-4593-46a6-9a8e-3935e09ece8f admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:28:48.764558 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b388bcfc-a7d5-45e6-966c-b7d05b0c8b8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:28:48.765172 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b388bcfc-a7d5-45e6-966c-b7d05b0c8b8d service cinder] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:28:48.765452 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b388bcfc-a7d5-45e6-966c-b7d05b0c8b8d service cinder] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:28:48.765728 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b388bcfc-a7d5-45e6-966c-b7d05b0c8b8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:28:48.774117 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b388bcfc-a7d5-45e6-966c-b7d05b0c8b8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:28:48.775594 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b388bcfc-a7d5-45e6-966c-b7d05b0c8b8d service cinder] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:28:48.787215 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1347/2711] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:28:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:28:58.687517 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=53,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:28:58.780257 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:INSERT=1,keystone:SELECT=28 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:29:15.847938 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9469538f-adb1-4e5c-9933-751a1d4257f4 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:29:15.909082 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9469538f-adb1-4e5c-9933-751a1d4257f4 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:29:15.913574 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b10ff0-347f-4709-b6af-2c2984884893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:29:15.914223 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b10ff0-347f-4709-b6af-2c2984884893 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:29:15.914503 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b10ff0-347f-4709-b6af-2c2984884893 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:29:15.914774 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b10ff0-347f-4709-b6af-2c2984884893 admin admin] PATH_INFO: `/v3/users/10da2e2e93014fef9053f8870005a717` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:29:15.915922 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b10ff0-347f-4709-b6af-2c2984884893 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10da2e2e93014fef9053f8870005a717)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:29:15.917814 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b10ff0-347f-4709-b6af-2c2984884893 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:29:16.070845 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-b0b10ff0-347f-4709-b6af-2c2984884893 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10da2e2e93014fef9053f8870005a717'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:29:16.148827 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-b0b10ff0-347f-4709-b6af-2c2984884893 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10da2e2e93014fef9053f8870005a717'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:29:16.149489 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-b0b10ff0-347f-4709-b6af-2c2984884893 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10da2e2e93014fef9053f8870005a717'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:29:16.150094 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-b0b10ff0-347f-4709-b6af-2c2984884893 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10da2e2e93014fef9053f8870005a717'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:29:16.157871 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-b0b10ff0-347f-4709-b6af-2c2984884893 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10da2e2e93014fef9053f8870005a717'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:29:16.175378 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1365/2712] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:29:15 2022] DELETE /identity/v3/users/10da2e2e93014fef9053f8870005a717 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:29:16.180089 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11383062-487a-45ac-9a05-dc70ee6322d8 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:29:16.181847 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11383062-487a-45ac-9a05-dc70ee6322d8 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:29:16.181847 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11383062-487a-45ac-9a05-dc70ee6322d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:29:16.200256 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-11383062-487a-45ac-9a05-dc70ee6322d8 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:29:16.201438 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-11383062-487a-45ac-9a05-dc70ee6322d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:29:16.260239 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1348/2713] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:29:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:29:16.270006 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b10ff0-347f-4709-b6af-2c2984884893 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:29:16.271450 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-b0b10ff0-347f-4709-b6af-2c2984884893 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:29:16.330140 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b10ff0-347f-4709-b6af-2c2984884893 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:29:16.333667 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d9816d-8c72-4a0e-af62-ad2d6c03cd39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:29:16.334292 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d9816d-8c72-4a0e-af62-ad2d6c03cd39 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:29:16.334569 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d9816d-8c72-4a0e-af62-ad2d6c03cd39 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:29:16.334838 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d9816d-8c72-4a0e-af62-ad2d6c03cd39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:29:16.344195 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d9816d-8c72-4a0e-af62-ad2d6c03cd39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:29:16.345634 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d9816d-8c72-4a0e-af62-ad2d6c03cd39 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:29:16.354930 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1366/2714] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:29:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:29:16.757324 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11383062-487a-45ac-9a05-dc70ee6322d8 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:29:16.758779 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-11383062-487a-45ac-9a05-dc70ee6322d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:29:16.772617 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11383062-487a-45ac-9a05-dc70ee6322d8 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:29:16.777574 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38bcae7-553e-427f-bc29-be7d3bd010dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:29:16.778289 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38bcae7-553e-427f-bc29-be7d3bd010dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:29:16.778579 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38bcae7-553e-427f-bc29-be7d3bd010dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:29:16.778856 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38bcae7-553e-427f-bc29-be7d3bd010dc admin admin] PATH_INFO: `/v3/projects/40133fcf86b84853868830b693a6b9e2` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:29:16.783313 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38bcae7-553e-427f-bc29-be7d3bd010dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=40133fcf86b84853868830b693a6b9e2)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:29:16.783313 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38bcae7-553e-427f-bc29-be7d3bd010dc admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:29:16.887364 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-f38bcae7-553e-427f-bc29-be7d3bd010dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40133fcf86b84853868830b693a6b9e2', 'request_id': 'req-f38bcae7-553e-427f-bc29-be7d3bd010dc'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:29:16.944569 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-f38bcae7-553e-427f-bc29-be7d3bd010dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40133fcf86b84853868830b693a6b9e2', 'request_id': 'req-f38bcae7-553e-427f-bc29-be7d3bd010dc'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:29:16.950343 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-f38bcae7-553e-427f-bc29-be7d3bd010dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40133fcf86b84853868830b693a6b9e2', 'request_id': 'req-f38bcae7-553e-427f-bc29-be7d3bd010dc'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:29:16.962085 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-f38bcae7-553e-427f-bc29-be7d3bd010dc admin admin] The token cache is being invalidate because project 40133fcf86b84853868830b693a6b9e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81650) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:29:16.962398 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-f38bcae7-553e-427f-bc29-be7d3bd010dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:29:16.963074 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-f38bcae7-553e-427f-bc29-be7d3bd010dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:29:16.964538 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1349/2715] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:29:16 2022] DELETE /identity/v3/projects/40133fcf86b84853868830b693a6b9e2 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:29:16.969640 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee2ff4c-b530-475f-b4dc-881a4608b54a None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:29:16.970116 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee2ff4c-b530-475f-b4dc-881a4608b54a None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:29:16.970227 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee2ff4c-b530-475f-b4dc-881a4608b54a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:29:16.989369 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-0ee2ff4c-b530-475f-b4dc-881a4608b54a None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:29:16.990561 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-0ee2ff4c-b530-475f-b4dc-881a4608b54a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:29:17.007397 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:29:17.051093 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1367/2716] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:29:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:29:17.059193 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c1faac-0ce2-4a45-bca5-7ff4deb56f0a None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:29:17.059498 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c1faac-0ce2-4a45-bca5-7ff4deb56f0a None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:29:17.059751 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c1faac-0ce2-4a45-bca5-7ff4deb56f0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:29:17.080472 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-14c1faac-0ce2-4a45-bca5-7ff4deb56f0a None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:29:17.081665 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-14c1faac-0ce2-4a45-bca5-7ff4deb56f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:29:17.089483 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1350/2717] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:29:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 17:29:17.093729 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee2ff4c-b530-475f-b4dc-881a4608b54a None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:29:17.103418 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee2ff4c-b530-475f-b4dc-881a4608b54a None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:29:17.108587 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f19469-2649-4f64-8996-38ea96cb2ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:29:17.109246 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f19469-2649-4f64-8996-38ea96cb2ba8 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:29:17.109522 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f19469-2649-4f64-8996-38ea96cb2ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:29:17.109792 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f19469-2649-4f64-8996-38ea96cb2ba8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:29:17.110432 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f19469-2649-4f64-8996-38ea96cb2ba8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:29:17.110731 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f19469-2649-4f64-8996-38ea96cb2ba8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:29:17.112345 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f19469-2649-4f64-8996-38ea96cb2ba8 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:29:17.124574 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1368/2718] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:29:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 17:29:17.129258 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c1faac-0ce2-4a45-bca5-7ff4deb56f0a None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:29:17.138672 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c1faac-0ce2-4a45-bca5-7ff4deb56f0a None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:29:17.143375 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee371ae-fd34-46f2-9267-ff327dedba4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:29:17.143968 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee371ae-fd34-46f2-9267-ff327dedba4d admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:29:17.144278 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee371ae-fd34-46f2-9267-ff327dedba4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:29:17.144556 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee371ae-fd34-46f2-9267-ff327dedba4d admin admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:29:17.145167 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee371ae-fd34-46f2-9267-ff327dedba4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:29:17.147126 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee371ae-fd34-46f2-9267-ff327dedba4d admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:29:17.237682 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1351/2719] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:29:17 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:29:17.241756 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f19469-2649-4f64-8996-38ea96cb2ba8 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:29:17.252354 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f19469-2649-4f64-8996-38ea96cb2ba8 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:29:17.308232 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e7ed0-43d8-436f-a235-a2ea3b65337a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:29:17.308834 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e7ed0-43d8-436f-a235-a2ea3b65337a admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:29:17.309115 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e7ed0-43d8-436f-a235-a2ea3b65337a admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:29:17.309391 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e7ed0-43d8-436f-a235-a2ea3b65337a admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:29:17.310016 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4e7ed0-43d8-436f-a235-a2ea3b65337a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:29:17.312302 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4e7ed0-43d8-436f-a235-a2ea3b65337a admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:29:17.391825 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1369/2720] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:29:17 2022] POST /identity/v3/users => generated 293 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:29:17.395832 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee371ae-fd34-46f2-9267-ff327dedba4d admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:29:17.407217 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee371ae-fd34-46f2-9267-ff327dedba4d admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:29:17.411748 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac2891c-a005-48dd-ab82-cf29f8ef0f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:29:17.412337 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac2891c-a005-48dd-ab82-cf29f8ef0f94 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:29:17.412615 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac2891c-a005-48dd-ab82-cf29f8ef0f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:29:17.412885 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac2891c-a005-48dd-ab82-cf29f8ef0f94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:29:17.413507 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac2891c-a005-48dd-ab82-cf29f8ef0f94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:29:17.415125 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac2891c-a005-48dd-ab82-cf29f8ef0f94 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:29:17.424819 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1352/2721] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:29:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:29:17.428649 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e7ed0-43d8-436f-a235-a2ea3b65337a admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:29:17.439761 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e7ed0-43d8-436f-a235-a2ea3b65337a admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:29:17.444412 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9ac199-d1f7-42b9-a2d0-0730850dd6ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:29:17.445008 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9ac199-d1f7-42b9-a2d0-0730850dd6ae admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:29:17.445285 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9ac199-d1f7-42b9-a2d0-0730850dd6ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:29:17.445559 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9ac199-d1f7-42b9-a2d0-0730850dd6ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:29:17.446184 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9ac199-d1f7-42b9-a2d0-0730850dd6ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:29:17.447795 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9ac199-d1f7-42b9-a2d0-0730850dd6ae admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:29:17.462448 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1370/2722] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:29:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:29:17.463812 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac2891c-a005-48dd-ab82-cf29f8ef0f94 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:29:17.473682 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac2891c-a005-48dd-ab82-cf29f8ef0f94 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:29:17.478227 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779793cd-1ae6-45fe-a1d2-44af80b580bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:29:17.478870 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779793cd-1ae6-45fe-a1d2-44af80b580bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:29:17.479185 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779793cd-1ae6-45fe-a1d2-44af80b580bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:29:17.479460 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779793cd-1ae6-45fe-a1d2-44af80b580bc admin admin] PATH_INFO: `/v3/projects/a06430b1fce94c91888e91a6b70722b0/users/f268ed5a5f6748718c0d74f2bdb037ce/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:29:17.493543 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779793cd-1ae6-45fe-a1d2-44af80b580bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=a06430b1fce94c91888e91a6b70722b0, user_id=f268ed5a5f6748718c0d74f2bdb037ce, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:29:17.496218 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779793cd-1ae6-45fe-a1d2-44af80b580bc admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:29:17.550159 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1353/2723] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:29:17 2022] PUT /identity/v3/projects/a06430b1fce94c91888e91a6b70722b0/users/f268ed5a5f6748718c0d74f2bdb037ce/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:29:17.561510 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b738a88-a96e-4310-a84c-cef0909cd676 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:29:17.561894 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b738a88-a96e-4310-a84c-cef0909cd676 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:29:17.562176 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b738a88-a96e-4310-a84c-cef0909cd676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:29:17.590256 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-3b738a88-a96e-4310-a84c-cef0909cd676 None None] MFA Rules not processed for user `f268ed5a5f6748718c0d74f2bdb037ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:29:17.591496 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-3b738a88-a96e-4310-a84c-cef0909cd676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:29:17.654406 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1371/2724] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:29:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:29:17.665482 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779793cd-1ae6-45fe-a1d2-44af80b580bc admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:29:17.667214 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-779793cd-1ae6-45fe-a1d2-44af80b580bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:29:17.690076 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:29:17.725433 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779793cd-1ae6-45fe-a1d2-44af80b580bc admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:29:17.729203 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de56ce65-0547-44ba-8086-4c0ed2e2e8e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:29:17.729813 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de56ce65-0547-44ba-8086-4c0ed2e2e8e0 service cinder] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:29:17.730099 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de56ce65-0547-44ba-8086-4c0ed2e2e8e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:29:17.730377 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de56ce65-0547-44ba-8086-4c0ed2e2e8e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:29:17.739247 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de56ce65-0547-44ba-8086-4c0ed2e2e8e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:29:17.740721 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de56ce65-0547-44ba-8086-4c0ed2e2e8e0 service cinder] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:29:17.750127 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1354/2725] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:29:17 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:29:20.821128 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b738a88-a96e-4310-a84c-cef0909cd676 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:29:20.839696 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:29:20.884442 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b738a88-a96e-4310-a84c-cef0909cd676 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:29:20.889088 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ff3439-4580-40a1-b1bb-879809f75196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:29:20.889870 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ff3439-4580-40a1-b1bb-879809f75196 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:29:20.890148 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ff3439-4580-40a1-b1bb-879809f75196 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:29:20.890420 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ff3439-4580-40a1-b1bb-879809f75196 admin admin] PATH_INFO: `/v3/users/f268ed5a5f6748718c0d74f2bdb037ce` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:29:20.891566 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ff3439-4580-40a1-b1bb-879809f75196 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f268ed5a5f6748718c0d74f2bdb037ce)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:29:20.893459 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ff3439-4580-40a1-b1bb-879809f75196 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:29:21.037552 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-87ff3439-4580-40a1-b1bb-879809f75196 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f268ed5a5f6748718c0d74f2bdb037ce'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:29:21.098781 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-87ff3439-4580-40a1-b1bb-879809f75196 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f268ed5a5f6748718c0d74f2bdb037ce'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:29:21.099480 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-87ff3439-4580-40a1-b1bb-879809f75196 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f268ed5a5f6748718c0d74f2bdb037ce'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:29:21.100136 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-87ff3439-4580-40a1-b1bb-879809f75196 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f268ed5a5f6748718c0d74f2bdb037ce'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:29:21.107265 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-87ff3439-4580-40a1-b1bb-879809f75196 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f268ed5a5f6748718c0d74f2bdb037ce'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:29:21.125556 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1372/2726] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:29:20 2022] DELETE /identity/v3/users/f268ed5a5f6748718c0d74f2bdb037ce => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:29:21.130095 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89768dea-6334-4700-bfed-1e965469f4bc None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:29:21.130589 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89768dea-6334-4700-bfed-1e965469f4bc None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:29:21.130688 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89768dea-6334-4700-bfed-1e965469f4bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:29:21.149266 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-89768dea-6334-4700-bfed-1e965469f4bc None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:29:21.150493 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-89768dea-6334-4700-bfed-1e965469f4bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:29:21.208186 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1355/2727] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:29:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:29:21.218321 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ff3439-4580-40a1-b1bb-879809f75196 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:29:21.219803 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-87ff3439-4580-40a1-b1bb-879809f75196 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:29:21.275658 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ff3439-4580-40a1-b1bb-879809f75196 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:29:21.279288 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f3ee2d-b83a-45c1-beeb-82ccb4f3c44c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:29:21.280519 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f3ee2d-b83a-45c1-beeb-82ccb4f3c44c service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:29:21.280800 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f3ee2d-b83a-45c1-beeb-82ccb4f3c44c service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:29:21.281069 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f3ee2d-b83a-45c1-beeb-82ccb4f3c44c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:29:21.289881 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f3ee2d-b83a-45c1-beeb-82ccb4f3c44c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:29:21.291293 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f3ee2d-b83a-45c1-beeb-82ccb4f3c44c service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:29:21.301137 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1373/2728] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:29:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:29:21.677338 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89768dea-6334-4700-bfed-1e965469f4bc None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:29:21.680759 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-89768dea-6334-4700-bfed-1e965469f4bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:29:21.693304 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89768dea-6334-4700-bfed-1e965469f4bc None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:29:21.698065 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4c9fcc-ce66-47ae-a39a-6aed0145decc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:29:21.698702 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4c9fcc-ce66-47ae-a39a-6aed0145decc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:29:21.699018 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4c9fcc-ce66-47ae-a39a-6aed0145decc admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:29:21.699299 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4c9fcc-ce66-47ae-a39a-6aed0145decc admin admin] PATH_INFO: `/v3/projects/a06430b1fce94c91888e91a6b70722b0` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:29:21.700451 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4c9fcc-ce66-47ae-a39a-6aed0145decc admin admin] RBAC: Authorizing `identity:delete_project(project_id=a06430b1fce94c91888e91a6b70722b0)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:29:21.702887 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4c9fcc-ce66-47ae-a39a-6aed0145decc admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:29:21.797350 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-ca4c9fcc-ce66-47ae-a39a-6aed0145decc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a06430b1fce94c91888e91a6b70722b0', 'request_id': 'req-ca4c9fcc-ce66-47ae-a39a-6aed0145decc'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:29:21.858671 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-ca4c9fcc-ce66-47ae-a39a-6aed0145decc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a06430b1fce94c91888e91a6b70722b0', 'request_id': 'req-ca4c9fcc-ce66-47ae-a39a-6aed0145decc'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:29:21.864329 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-ca4c9fcc-ce66-47ae-a39a-6aed0145decc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a06430b1fce94c91888e91a6b70722b0', 'request_id': 'req-ca4c9fcc-ce66-47ae-a39a-6aed0145decc'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:29:21.875501 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-ca4c9fcc-ce66-47ae-a39a-6aed0145decc admin admin] The token cache is being invalidate because project a06430b1fce94c91888e91a6b70722b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81650) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:29:21.875789 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-ca4c9fcc-ce66-47ae-a39a-6aed0145decc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:29:21.876414 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-ca4c9fcc-ce66-47ae-a39a-6aed0145decc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:29:21.877907 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1356/2729] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:29:21 2022] DELETE /identity/v3/projects/a06430b1fce94c91888e91a6b70722b0 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:29:21.882767 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca7f7c-394a-4676-9efb-dd6b6f74166d None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:29:21.883309 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca7f7c-394a-4676-9efb-dd6b6f74166d None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:29:21.883309 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca7f7c-394a-4676-9efb-dd6b6f74166d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:29:21.903074 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-eaca7f7c-394a-4676-9efb-dd6b6f74166d None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:29:21.904259 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-eaca7f7c-394a-4676-9efb-dd6b6f74166d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:29:21.938242 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:29:21.962711 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1374/2730] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:29:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:29:21.971850 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800d1e8c-a527-4ec9-b299-6cf96718923c None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:29:21.974970 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800d1e8c-a527-4ec9-b299-6cf96718923c None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:29:21.974970 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800d1e8c-a527-4ec9-b299-6cf96718923c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:29:21.992346 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-800d1e8c-a527-4ec9-b299-6cf96718923c None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:29:21.993546 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-800d1e8c-a527-4ec9-b299-6cf96718923c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:29:22.002360 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1357/2731] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:29:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:29:22.006855 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca7f7c-394a-4676-9efb-dd6b6f74166d None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:29:22.017167 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca7f7c-394a-4676-9efb-dd6b6f74166d None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:29:22.021770 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ecd0d3-25d1-4e26-89bf-d80214c7b0b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:29:22.022369 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ecd0d3-25d1-4e26-89bf-d80214c7b0b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:29:22.022651 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ecd0d3-25d1-4e26-89bf-d80214c7b0b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:29:22.022951 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ecd0d3-25d1-4e26-89bf-d80214c7b0b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:29:22.023595 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ecd0d3-25d1-4e26-89bf-d80214c7b0b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:29:22.023898 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ecd0d3-25d1-4e26-89bf-d80214c7b0b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:29:22.025489 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ecd0d3-25d1-4e26-89bf-d80214c7b0b2 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:29:22.036789 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1375/2732] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:29:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 17:29:22.041197 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800d1e8c-a527-4ec9-b299-6cf96718923c None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:29:22.050888 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800d1e8c-a527-4ec9-b299-6cf96718923c None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:29:22.055628 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723fd44b-ce0f-4cd8-a238-d3ed6c34f9a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:29:22.056228 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723fd44b-ce0f-4cd8-a238-d3ed6c34f9a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:29:22.056510 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723fd44b-ce0f-4cd8-a238-d3ed6c34f9a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:29:22.056800 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723fd44b-ce0f-4cd8-a238-d3ed6c34f9a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:29:22.057719 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723fd44b-ce0f-4cd8-a238-d3ed6c34f9a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:29:22.059671 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723fd44b-ce0f-4cd8-a238-d3ed6c34f9a3 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:29:22.183397 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1358/2733] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:29:22 2022] POST /identity/v3/projects => generated 375 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:29:22.189410 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ecd0d3-25d1-4e26-89bf-d80214c7b0b2 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:29:22.201206 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ecd0d3-25d1-4e26-89bf-d80214c7b0b2 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:29:22.265831 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b55cf16-f149-4559-87fd-a8dccfd90512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:29:22.266430 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b55cf16-f149-4559-87fd-a8dccfd90512 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:29:22.266709 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b55cf16-f149-4559-87fd-a8dccfd90512 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:29:22.267015 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b55cf16-f149-4559-87fd-a8dccfd90512 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:29:22.267630 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b55cf16-f149-4559-87fd-a8dccfd90512 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:29:22.269531 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b55cf16-f149-4559-87fd-a8dccfd90512 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:29:22.361653 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1376/2734] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:29:22 2022] POST /identity/v3/users => generated 288 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:29:22.365789 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723fd44b-ce0f-4cd8-a238-d3ed6c34f9a3 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:29:22.377440 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723fd44b-ce0f-4cd8-a238-d3ed6c34f9a3 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:29:22.382029 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec221e6-fa01-40b4-8130-9be0938604f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:29:22.382632 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec221e6-fa01-40b4-8130-9be0938604f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:29:22.382999 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec221e6-fa01-40b4-8130-9be0938604f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:29:22.383273 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec221e6-fa01-40b4-8130-9be0938604f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:29:22.383928 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec221e6-fa01-40b4-8130-9be0938604f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:29:22.385546 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec221e6-fa01-40b4-8130-9be0938604f3 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:29:22.395636 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1359/2735] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:29:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:29:22.399582 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b55cf16-f149-4559-87fd-a8dccfd90512 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:29:22.409600 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b55cf16-f149-4559-87fd-a8dccfd90512 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:29:22.414211 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf734ecc-ae8a-46f7-af55-cab89464e65d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:29:22.414809 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf734ecc-ae8a-46f7-af55-cab89464e65d admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:29:22.415131 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf734ecc-ae8a-46f7-af55-cab89464e65d admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:29:22.415405 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf734ecc-ae8a-46f7-af55-cab89464e65d admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:29:22.416028 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf734ecc-ae8a-46f7-af55-cab89464e65d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:29:22.417646 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf734ecc-ae8a-46f7-af55-cab89464e65d admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:29:22.428669 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1377/2736] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:29:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:29:22.433006 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec221e6-fa01-40b4-8130-9be0938604f3 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:29:22.443796 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec221e6-fa01-40b4-8130-9be0938604f3 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:29:22.448240 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ec1dc8-e229-4b7c-a83c-bfa951ab1a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:29:22.448884 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec1dc8-e229-4b7c-a83c-bfa951ab1a4f admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:29:22.449163 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec1dc8-e229-4b7c-a83c-bfa951ab1a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:29:22.449438 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec1dc8-e229-4b7c-a83c-bfa951ab1a4f admin admin] PATH_INFO: `/v3/projects/596a9a6708ff4cffb91ebb767bc1de6c/users/4cc76ea92fcb4686a87006880fef657c/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:29:22.451621 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:29:22.464061 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ec1dc8-e229-4b7c-a83c-bfa951ab1a4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=596a9a6708ff4cffb91ebb767bc1de6c, user_id=4cc76ea92fcb4686a87006880fef657c, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:29:22.466750 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ec1dc8-e229-4b7c-a83c-bfa951ab1a4f admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:29:22.538397 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1360/2737] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:29:22 2022] PUT /identity/v3/projects/596a9a6708ff4cffb91ebb767bc1de6c/users/4cc76ea92fcb4686a87006880fef657c/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:29:22.549603 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed2990c-4883-4f4c-a811-56227a0f5c5b None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:29:22.550433 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed2990c-4883-4f4c-a811-56227a0f5c5b None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:29:22.551197 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed2990c-4883-4f4c-a811-56227a0f5c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:29:22.580855 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-2ed2990c-4883-4f4c-a811-56227a0f5c5b None None] MFA Rules not processed for user `4cc76ea92fcb4686a87006880fef657c`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:29:22.582099 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-2ed2990c-4883-4f4c-a811-56227a0f5c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:29:22.643757 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1378/2738] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:29:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:29:22.655133 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ec1dc8-e229-4b7c-a83c-bfa951ab1a4f admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:29:22.656823 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-22ec1dc8-e229-4b7c-a83c-bfa951ab1a4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:29:22.713029 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ec1dc8-e229-4b7c-a83c-bfa951ab1a4f admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:29:22.716625 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134e5825-1505-4eb0-8ad2-a9c67b2366ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:29:22.717720 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134e5825-1505-4eb0-8ad2-a9c67b2366ba service cinder] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:29:22.717990 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134e5825-1505-4eb0-8ad2-a9c67b2366ba service cinder] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:29:22.718256 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134e5825-1505-4eb0-8ad2-a9c67b2366ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:29:22.726343 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134e5825-1505-4eb0-8ad2-a9c67b2366ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:29:22.727815 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134e5825-1505-4eb0-8ad2-a9c67b2366ba service cinder] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:29:22.737177 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1361/2739] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:29:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:29:24.441943 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ddc7b-9e1a-416c-a83b-0fc224d8d68b None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:29:24.442481 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ddc7b-9e1a-416c-a83b-0fc224d8d68b None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:29:24.442481 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ddc7b-9e1a-416c-a83b-0fc224d8d68b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:29:24.461314 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-fd8ddc7b-9e1a-416c-a83b-0fc224d8d68b None None] MFA Rules not processed for user `4cc76ea92fcb4686a87006880fef657c`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:29:24.462513 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-fd8ddc7b-9e1a-416c-a83b-0fc224d8d68b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:29:24.470051 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1379/2740] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:29:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:29:25.030112 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134e5825-1505-4eb0-8ad2-a9c67b2366ba service cinder] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:29:25.031666 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-134e5825-1505-4eb0-8ad2-a9c67b2366ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:29:25.104830 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134e5825-1505-4eb0-8ad2-a9c67b2366ba service cinder] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:29:25.110520 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45340ae2-67bb-4567-a3c5-33e1a32b034f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29653ba1a2a74232972ef7a9000518b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:29:25.111178 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45340ae2-67bb-4567-a3c5-33e1a32b034f service placement] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:29:25.111460 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45340ae2-67bb-4567-a3c5-33e1a32b034f service placement] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:29:25.111732 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45340ae2-67bb-4567-a3c5-33e1a32b034f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:29:25.113443 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-45340ae2-67bb-4567-a3c5-33e1a32b034f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:29:25.125549 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45340ae2-67bb-4567-a3c5-33e1a32b034f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:29:25.127163 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45340ae2-67bb-4567-a3c5-33e1a32b034f service placement] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:29:25.136826 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1362/2741] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:29:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:29:34.456055 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=84,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:29:35.132672 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=65,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:29:35.650197 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8ddc7b-9e1a-416c-a83b-0fc224d8d68b None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:29:35.652455 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-fd8ddc7b-9e1a-416c-a83b-0fc224d8d68b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:29:35.730719 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8ddc7b-9e1a-416c-a83b-0fc224d8d68b None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:29:35.736846 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a12dad2-2acb-4b54-a78f-e68a22320d34 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55723c756cae4e7ab636054db48de02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:29:35.737462 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a12dad2-2acb-4b54-a78f-e68a22320d34 service nova] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:29:35.737745 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a12dad2-2acb-4b54-a78f-e68a22320d34 service nova] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:29:35.739034 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a12dad2-2acb-4b54-a78f-e68a22320d34 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:29:35.739841 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-4a12dad2-2acb-4b54-a78f-e68a22320d34 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:29:35.752561 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a12dad2-2acb-4b54-a78f-e68a22320d34 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:29:35.754185 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a12dad2-2acb-4b54-a78f-e68a22320d34 service nova] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:29:35.767088 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1380/2742] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:29:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:29:42.114031 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faef04d2-082a-46f4-a9ed-2ac42b39c5aa None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:29:42.114493 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faef04d2-082a-46f4-a9ed-2ac42b39c5aa None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:29:42.114600 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faef04d2-082a-46f4-a9ed-2ac42b39c5aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:29:42.134505 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-faef04d2-082a-46f4-a9ed-2ac42b39c5aa None None] MFA Rules not processed for user `5de19a54ed104bcdb35ee4e5e793f1e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:29:42.135761 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-faef04d2-082a-46f4-a9ed-2ac42b39c5aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:29:42.190377 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1363/2743] 10.222.0.82 () {58 vars in 1009 bytes} [Thu Aug 18 17:29:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:29:42.325280 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a12dad2-2acb-4b54-a78f-e68a22320d34 service nova] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:29:42.326672 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-4a12dad2-2acb-4b54-a78f-e68a22320d34 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:29:42.405553 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a12dad2-2acb-4b54-a78f-e68a22320d34 service nova] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:29:42.411209 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a543f993-74cf-4caf-b65b-390ec4f87eb5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1381673f81584f4b8084823e3aa99e2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:29:42.411807 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a543f993-74cf-4caf-b65b-390ec4f87eb5 service swift] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:29:42.412082 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a543f993-74cf-4caf-b65b-390ec4f87eb5 service swift] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:29:42.412352 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a543f993-74cf-4caf-b65b-390ec4f87eb5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:29:42.420115 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a543f993-74cf-4caf-b65b-390ec4f87eb5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:29:42.422263 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a543f993-74cf-4caf-b65b-390ec4f87eb5 service swift] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:29:42.430076 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1381/2744] 10.222.0.82 () {60 vars in 1384 bytes} [Thu Aug 18 17:29:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 17:29:43.250517 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a43d15-0ab9-4fc0-b945-51a83e8bf13c None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:29:43.251057 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a43d15-0ab9-4fc0-b945-51a83e8bf13c None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:29:43.251163 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a43d15-0ab9-4fc0-b945-51a83e8bf13c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:29:43.270076 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-00a43d15-0ab9-4fc0-b945-51a83e8bf13c None None] MFA Rules not processed for user `5de19a54ed104bcdb35ee4e5e793f1e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:29:43.271313 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-00a43d15-0ab9-4fc0-b945-51a83e8bf13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:29:43.278327 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1364/2745] 10.222.0.82 () {58 vars in 967 bytes} [Thu Aug 18 17:29:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:29:43.291115 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a543f993-74cf-4caf-b65b-390ec4f87eb5 service swift] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:29:43.301767 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a543f993-74cf-4caf-b65b-390ec4f87eb5 service swift] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:29:43.307138 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc626f8-6404-4e63-ba74-da3c03ec38c2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1381673f81584f4b8084823e3aa99e2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:29:43.307729 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc626f8-6404-4e63-ba74-da3c03ec38c2 service swift] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:29:43.308012 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc626f8-6404-4e63-ba74-da3c03ec38c2 service swift] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:29:43.308278 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc626f8-6404-4e63-ba74-da3c03ec38c2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:29:43.316400 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc626f8-6404-4e63-ba74-da3c03ec38c2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:29:43.317963 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc626f8-6404-4e63-ba74-da3c03ec38c2 service swift] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:29:43.326285 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1382/2746] 10.222.0.82 () {60 vars in 1384 bytes} [Thu Aug 18 17:29:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 17:29:43.868054 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48445d4b-97c7-421b-83ba-af0be30dd227 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:29:43.868343 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48445d4b-97c7-421b-83ba-af0be30dd227 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:29:43.868622 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48445d4b-97c7-421b-83ba-af0be30dd227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:29:43.887574 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-48445d4b-97c7-421b-83ba-af0be30dd227 None None] MFA Rules not processed for user `5de19a54ed104bcdb35ee4e5e793f1e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:29:43.888753 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-48445d4b-97c7-421b-83ba-af0be30dd227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:29:43.896481 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1365/2747] 10.222.0.82 () {58 vars in 1009 bytes} [Thu Aug 18 17:29:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:29:43.908394 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc626f8-6404-4e63-ba74-da3c03ec38c2 service swift] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:29:43.919163 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc626f8-6404-4e63-ba74-da3c03ec38c2 service swift] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:29:43.924412 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debb2d80-6b93-413e-8f13-002a7246302e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1381673f81584f4b8084823e3aa99e2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:29:43.925016 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb2d80-6b93-413e-8f13-002a7246302e service swift] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:29:43.925273 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb2d80-6b93-413e-8f13-002a7246302e service swift] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:29:43.925544 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb2d80-6b93-413e-8f13-002a7246302e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:29:43.933567 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debb2d80-6b93-413e-8f13-002a7246302e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:29:43.935153 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debb2d80-6b93-413e-8f13-002a7246302e service swift] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:29:43.943687 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1383/2748] 10.222.0.82 () {60 vars in 1384 bytes} [Thu Aug 18 17:29:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 17:29:44.498640 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da005d-41e0-4584-8e57-89db31a1492a None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:29:44.499609 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da005d-41e0-4584-8e57-89db31a1492a None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:29:44.499927 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da005d-41e0-4584-8e57-89db31a1492a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:29:44.519159 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-01da005d-41e0-4584-8e57-89db31a1492a None None] MFA Rules not processed for user `5de19a54ed104bcdb35ee4e5e793f1e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:29:44.520366 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-01da005d-41e0-4584-8e57-89db31a1492a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:29:44.527746 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1366/2749] 10.222.0.82 () {58 vars in 967 bytes} [Thu Aug 18 17:29:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:29:44.540508 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debb2d80-6b93-413e-8f13-002a7246302e service swift] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:29:44.551833 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debb2d80-6b93-413e-8f13-002a7246302e service swift] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:29:44.557377 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de785476-f716-4e75-9942-36929a55bfb5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1381673f81584f4b8084823e3aa99e2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:29:44.558020 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de785476-f716-4e75-9942-36929a55bfb5 service swift] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:29:44.558307 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de785476-f716-4e75-9942-36929a55bfb5 service swift] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:29:44.558640 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de785476-f716-4e75-9942-36929a55bfb5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:29:44.567239 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de785476-f716-4e75-9942-36929a55bfb5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:29:44.568815 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de785476-f716-4e75-9942-36929a55bfb5 service swift] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:29:44.576982 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1384/2750] 10.222.0.82 () {60 vars in 1384 bytes} [Thu Aug 18 17:29:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 17:29:47.269806 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01da005d-41e0-4584-8e57-89db31a1492a None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:29:47.331255 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01da005d-41e0-4584-8e57-89db31a1492a None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:29:47.335675 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a86df85-2135-4d12-9da2-710b421e5c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:29:47.336295 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a86df85-2135-4d12-9da2-710b421e5c8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:29:47.336581 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a86df85-2135-4d12-9da2-710b421e5c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:29:47.336854 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a86df85-2135-4d12-9da2-710b421e5c8e admin admin] PATH_INFO: `/v3/users/4cc76ea92fcb4686a87006880fef657c` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:29:47.337967 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a86df85-2135-4d12-9da2-710b421e5c8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cc76ea92fcb4686a87006880fef657c)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:29:47.339906 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a86df85-2135-4d12-9da2-710b421e5c8e admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:29:47.511110 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-6a86df85-2135-4d12-9da2-710b421e5c8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cc76ea92fcb4686a87006880fef657c'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:29:47.522920 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-6a86df85-2135-4d12-9da2-710b421e5c8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cc76ea92fcb4686a87006880fef657c'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:29:47.523615 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-6a86df85-2135-4d12-9da2-710b421e5c8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cc76ea92fcb4686a87006880fef657c'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:29:47.572073 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-6a86df85-2135-4d12-9da2-710b421e5c8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cc76ea92fcb4686a87006880fef657c'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:29:47.572700 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-6a86df85-2135-4d12-9da2-710b421e5c8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cc76ea92fcb4686a87006880fef657c'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:29:47.585022 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1367/2751] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:29:47 2022] DELETE /identity/v3/users/4cc76ea92fcb4686a87006880fef657c => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:29:47.589766 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd1587-ba9f-4ccd-9530-9361d7b61539 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:29:47.590228 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd1587-ba9f-4ccd-9530-9361d7b61539 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:29:47.590342 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd1587-ba9f-4ccd-9530-9361d7b61539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:29:47.609429 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-63cd1587-ba9f-4ccd-9530-9361d7b61539 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:29:47.611276 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-63cd1587-ba9f-4ccd-9530-9361d7b61539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:29:47.629141 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:29:47.669802 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1385/2752] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:29:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:29:47.680308 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a86df85-2135-4d12-9da2-710b421e5c8e admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:29:47.682001 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-6a86df85-2135-4d12-9da2-710b421e5c8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:29:47.688872 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:29:47.740430 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a86df85-2135-4d12-9da2-710b421e5c8e admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:29:47.744539 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c554cf00-1aff-44a0-b20c-0b986a523f18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:29:47.745139 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c554cf00-1aff-44a0-b20c-0b986a523f18 service neutron] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:29:47.761085 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c554cf00-1aff-44a0-b20c-0b986a523f18 service neutron] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:29:47.761085 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c554cf00-1aff-44a0-b20c-0b986a523f18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:29:47.761085 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c554cf00-1aff-44a0-b20c-0b986a523f18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:29:47.761085 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c554cf00-1aff-44a0-b20c-0b986a523f18 service neutron] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:29:47.767960 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1368/2753] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:29:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:29:47.985580 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cd1587-ba9f-4ccd-9530-9361d7b61539 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:29:47.987127 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-63cd1587-ba9f-4ccd-9530-9361d7b61539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:29:48.000163 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cd1587-ba9f-4ccd-9530-9361d7b61539 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:29:48.005330 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204c51cc-a1f0-4493-9a59-1b62e40c93d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:29:48.005979 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204c51cc-a1f0-4493-9a59-1b62e40c93d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:29:48.006260 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204c51cc-a1f0-4493-9a59-1b62e40c93d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:29:48.006543 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204c51cc-a1f0-4493-9a59-1b62e40c93d2 admin admin] PATH_INFO: `/v3/projects/596a9a6708ff4cffb91ebb767bc1de6c` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:29:48.007712 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204c51cc-a1f0-4493-9a59-1b62e40c93d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=596a9a6708ff4cffb91ebb767bc1de6c)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:29:48.009613 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204c51cc-a1f0-4493-9a59-1b62e40c93d2 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:29:48.108528 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-204c51cc-a1f0-4493-9a59-1b62e40c93d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '596a9a6708ff4cffb91ebb767bc1de6c', 'request_id': 'req-204c51cc-a1f0-4493-9a59-1b62e40c93d2'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:29:48.114795 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-204c51cc-a1f0-4493-9a59-1b62e40c93d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '596a9a6708ff4cffb91ebb767bc1de6c', 'request_id': 'req-204c51cc-a1f0-4493-9a59-1b62e40c93d2'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:29:48.182628 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-204c51cc-a1f0-4493-9a59-1b62e40c93d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '596a9a6708ff4cffb91ebb767bc1de6c', 'request_id': 'req-204c51cc-a1f0-4493-9a59-1b62e40c93d2'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:29:48.193807 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-204c51cc-a1f0-4493-9a59-1b62e40c93d2 admin admin] The token cache is being invalidate because project 596a9a6708ff4cffb91ebb767bc1de6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:29:48.194101 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-204c51cc-a1f0-4493-9a59-1b62e40c93d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:29:48.194721 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-204c51cc-a1f0-4493-9a59-1b62e40c93d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:29:48.196291 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1386/2754] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:29:47 2022] DELETE /identity/v3/projects/596a9a6708ff4cffb91ebb767bc1de6c => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:29:57.777768 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=24 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:29:58.190801 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=43,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:29:59.393805 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c554cf00-1aff-44a0-b20c-0b986a523f18 service neutron] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:29:59.395260 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-c554cf00-1aff-44a0-b20c-0b986a523f18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:29:59.469156 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c554cf00-1aff-44a0-b20c-0b986a523f18 service neutron] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:29:59.531501 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4981ee94-04a5-4afe-b90d-8d775642b4f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55723c756cae4e7ab636054db48de02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:29:59.532625 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4981ee94-04a5-4afe-b90d-8d775642b4f1 service nova] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:29:59.532951 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4981ee94-04a5-4afe-b90d-8d775642b4f1 service nova] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:29:59.533234 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4981ee94-04a5-4afe-b90d-8d775642b4f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:29:59.535245 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-4981ee94-04a5-4afe-b90d-8d775642b4f1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:29:59.591236 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4981ee94-04a5-4afe-b90d-8d775642b4f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:29:59.592834 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4981ee94-04a5-4afe-b90d-8d775642b4f1 service nova] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:29:59.602507 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1369/2755] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:29:59 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:30:09.598266 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=84 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:30:19.264858 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a749d5bb-17f1-4b8f-a4e3-4aa90ff24acb None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:30:19.265330 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a749d5bb-17f1-4b8f-a4e3-4aa90ff24acb None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:30:19.265432 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a749d5bb-17f1-4b8f-a4e3-4aa90ff24acb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:30:19.285667 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-a749d5bb-17f1-4b8f-a4e3-4aa90ff24acb None None] MFA Rules not processed for user `da44ae3a4e8b41d3b8a3193db939e82d`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:30:19.286861 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-a749d5bb-17f1-4b8f-a4e3-4aa90ff24acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:30:19.293604 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1387/2756] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:30:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:30:29.280807 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:30:42.468182 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4981ee94-04a5-4afe-b90d-8d775642b4f1 service nova] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:30:42.469802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-4981ee94-04a5-4afe-b90d-8d775642b4f1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:30:42.604547 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4981ee94-04a5-4afe-b90d-8d775642b4f1 service nova] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:30:42.609905 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890a9795-55ae-4a2f-b3d5-7811eab37f99 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29653ba1a2a74232972ef7a9000518b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:30:42.610695 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890a9795-55ae-4a2f-b3d5-7811eab37f99 service placement] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:30:42.611010 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890a9795-55ae-4a2f-b3d5-7811eab37f99 service placement] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:30:42.611292 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890a9795-55ae-4a2f-b3d5-7811eab37f99 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:30:42.613050 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-890a9795-55ae-4a2f-b3d5-7811eab37f99 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:30:42.624443 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890a9795-55ae-4a2f-b3d5-7811eab37f99 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:30:42.626656 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890a9795-55ae-4a2f-b3d5-7811eab37f99 service placement] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:30:42.636841 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1370/2757] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:30:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:30:52.632326 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=63 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:31:55.287244 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a749d5bb-17f1-4b8f-a4e3-4aa90ff24acb None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:31:55.288790 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-a749d5bb-17f1-4b8f-a4e3-4aa90ff24acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:31:55.378595 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a749d5bb-17f1-4b8f-a4e3-4aa90ff24acb None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:31:55.386134 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8221f163-f087-42bf-a3b4-cdd93517d38c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55723c756cae4e7ab636054db48de02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:31:55.388184 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221f163-f087-42bf-a3b4-cdd93517d38c service nova] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:31:55.388184 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221f163-f087-42bf-a3b4-cdd93517d38c service nova] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:31:55.388184 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221f163-f087-42bf-a3b4-cdd93517d38c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:31:55.390191 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-8221f163-f087-42bf-a3b4-cdd93517d38c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:31:55.403695 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8221f163-f087-42bf-a3b4-cdd93517d38c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:31:55.405419 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8221f163-f087-42bf-a3b4-cdd93517d38c service nova] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:31:55.418866 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1388/2758] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:31:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:32:02.081758 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890a9795-55ae-4a2f-b3d5-7811eab37f99 service placement] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:02.083235 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-890a9795-55ae-4a2f-b3d5-7811eab37f99 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:32:02.159842 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890a9795-55ae-4a2f-b3d5-7811eab37f99 service placement] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:02.163719 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103f8833-0098-4ad4-8d96-5b0323ed0e3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:02.164317 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103f8833-0098-4ad4-8d96-5b0323ed0e3a service neutron] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:02.164603 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103f8833-0098-4ad4-8d96-5b0323ed0e3a service neutron] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:02.164876 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103f8833-0098-4ad4-8d96-5b0323ed0e3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:02.166634 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-103f8833-0098-4ad4-8d96-5b0323ed0e3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:32:02.253342 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103f8833-0098-4ad4-8d96-5b0323ed0e3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:02.254875 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103f8833-0098-4ad4-8d96-5b0323ed0e3a service neutron] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:02.275307 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1371/2759] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:32:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:32:05.412728 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=30 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:32:05.725199 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8221f163-f087-42bf-a3b4-cdd93517d38c service nova] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:05.726715 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-8221f163-f087-42bf-a3b4-cdd93517d38c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:32:05.742200 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8221f163-f087-42bf-a3b4-cdd93517d38c service nova] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:05.747440 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7baaf14-338a-441d-86ae-56ba5b782a07 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29653ba1a2a74232972ef7a9000518b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:05.748036 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7baaf14-338a-441d-86ae-56ba5b782a07 service placement] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:05.748315 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7baaf14-338a-441d-86ae-56ba5b782a07 service placement] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:05.748588 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7baaf14-338a-441d-86ae-56ba5b782a07 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:05.750287 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-d7baaf14-338a-441d-86ae-56ba5b782a07 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:32:05.762840 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7baaf14-338a-441d-86ae-56ba5b782a07 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:05.764671 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7baaf14-338a-441d-86ae-56ba5b782a07 service placement] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:05.776095 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1389/2760] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:32:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:32:05.836385 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103f8833-0098-4ad4-8d96-5b0323ed0e3a service neutron] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:05.837853 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-103f8833-0098-4ad4-8d96-5b0323ed0e3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:32:05.850025 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103f8833-0098-4ad4-8d96-5b0323ed0e3a service neutron] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:05.854789 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bd38b2-e887-49b2-82bc-e5ee3288a8a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:05.855453 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd38b2-e887-49b2-82bc-e5ee3288a8a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:05.855730 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd38b2-e887-49b2-82bc-e5ee3288a8a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:05.855999 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd38b2-e887-49b2-82bc-e5ee3288a8a7 admin admin] PATH_INFO: `/v3/users/da44ae3a4e8b41d3b8a3193db939e82d` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:05.857098 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bd38b2-e887-49b2-82bc-e5ee3288a8a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da44ae3a4e8b41d3b8a3193db939e82d)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:05.859141 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bd38b2-e887-49b2-82bc-e5ee3288a8a7 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:06.030756 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-f2bd38b2-e887-49b2-82bc-e5ee3288a8a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da44ae3a4e8b41d3b8a3193db939e82d'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:06.038194 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:32:06.043186 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-f2bd38b2-e887-49b2-82bc-e5ee3288a8a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da44ae3a4e8b41d3b8a3193db939e82d'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:06.044493 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-f2bd38b2-e887-49b2-82bc-e5ee3288a8a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da44ae3a4e8b41d3b8a3193db939e82d'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:06.114428 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-f2bd38b2-e887-49b2-82bc-e5ee3288a8a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da44ae3a4e8b41d3b8a3193db939e82d'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:06.115107 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-f2bd38b2-e887-49b2-82bc-e5ee3288a8a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da44ae3a4e8b41d3b8a3193db939e82d'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:06.127764 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1372/2761] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:32:05 2022] DELETE /identity/v3/users/da44ae3a4e8b41d3b8a3193db939e82d => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:32:06.428629 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7baaf14-338a-441d-86ae-56ba5b782a07 service placement] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:06.437491 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7baaf14-338a-441d-86ae-56ba5b782a07 service placement] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:06.495145 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3ec22c-6a00-428d-b4ed-751094bc0c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:06.495809 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3ec22c-6a00-428d-b4ed-751094bc0c20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:06.496088 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3ec22c-6a00-428d-b4ed-751094bc0c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:06.496388 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3ec22c-6a00-428d-b4ed-751094bc0c20 admin admin] PATH_INFO: `/v3/projects/808690fdba2741df889e0a21027606f9` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:06.497487 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3ec22c-6a00-428d-b4ed-751094bc0c20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=808690fdba2741df889e0a21027606f9)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:06.499429 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3ec22c-6a00-428d-b4ed-751094bc0c20 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:06.631462 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-7d3ec22c-6a00-428d-b4ed-751094bc0c20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '808690fdba2741df889e0a21027606f9', 'request_id': 'req-7d3ec22c-6a00-428d-b4ed-751094bc0c20'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:06.637885 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-7d3ec22c-6a00-428d-b4ed-751094bc0c20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '808690fdba2741df889e0a21027606f9', 'request_id': 'req-7d3ec22c-6a00-428d-b4ed-751094bc0c20'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:06.704672 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-7d3ec22c-6a00-428d-b4ed-751094bc0c20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '808690fdba2741df889e0a21027606f9', 'request_id': 'req-7d3ec22c-6a00-428d-b4ed-751094bc0c20'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:06.715287 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-7d3ec22c-6a00-428d-b4ed-751094bc0c20 admin admin] The token cache is being invalidate because project 808690fdba2741df889e0a21027606f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:32:06.715578 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-7d3ec22c-6a00-428d-b4ed-751094bc0c20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:06.716222 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-7d3ec22c-6a00-428d-b4ed-751094bc0c20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:06.717676 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1390/2762] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:32:06 2022] DELETE /identity/v3/projects/808690fdba2741df889e0a21027606f9 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:32:06.722524 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f1168-be8f-403a-b5ca-45c8afb24a98 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:06.722814 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f1168-be8f-403a-b5ca-45c8afb24a98 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:06.723634 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f1168-be8f-403a-b5ca-45c8afb24a98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:06.781775 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-b68f1168-be8f-403a-b5ca-45c8afb24a98 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:32:06.782928 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-b68f1168-be8f-403a-b5ca-45c8afb24a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:32:06.841647 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1373/2763] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:32:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:32:06.850060 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018e610-ae47-4a1b-a55a-677b41d32f91 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:06.850340 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018e610-ae47-4a1b-a55a-677b41d32f91 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:06.850642 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018e610-ae47-4a1b-a55a-677b41d32f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:06.870212 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-0018e610-ae47-4a1b-a55a-677b41d32f91 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:32:06.871432 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-0018e610-ae47-4a1b-a55a-677b41d32f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:32:06.879236 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1391/2764] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:32:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:32:06.883628 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f1168-be8f-403a-b5ca-45c8afb24a98 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:06.893303 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f1168-be8f-403a-b5ca-45c8afb24a98 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:06.898178 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fa117a-bbf5-4397-ba7c-839407f9b03d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:06.898772 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fa117a-bbf5-4397-ba7c-839407f9b03d admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:06.899090 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fa117a-bbf5-4397-ba7c-839407f9b03d admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:06.899365 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fa117a-bbf5-4397-ba7c-839407f9b03d admin admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:06.900002 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fa117a-bbf5-4397-ba7c-839407f9b03d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:32:06.900302 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fa117a-bbf5-4397-ba7c-839407f9b03d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:06.901893 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fa117a-bbf5-4397-ba7c-839407f9b03d admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:06.914134 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1374/2765] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:32:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 17:32:06.918170 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018e610-ae47-4a1b-a55a-677b41d32f91 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:06.928325 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018e610-ae47-4a1b-a55a-677b41d32f91 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:06.933052 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04407af-2ed7-4fcd-9264-69e62b7661b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:06.933646 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04407af-2ed7-4fcd-9264-69e62b7661b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:06.933926 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04407af-2ed7-4fcd-9264-69e62b7661b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:06.934224 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04407af-2ed7-4fcd-9264-69e62b7661b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:06.934843 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04407af-2ed7-4fcd-9264-69e62b7661b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:06.936751 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04407af-2ed7-4fcd-9264-69e62b7661b0 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:07.012771 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1392/2766] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:32:06 2022] POST /identity/v3/projects => generated 371 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:32:07.016934 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fa117a-bbf5-4397-ba7c-839407f9b03d admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:07.027577 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fa117a-bbf5-4397-ba7c-839407f9b03d admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:07.084681 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb72994c-7243-4980-a8e9-9996e045add1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:07.085312 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb72994c-7243-4980-a8e9-9996e045add1 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:07.085594 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb72994c-7243-4980-a8e9-9996e045add1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:07.085866 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb72994c-7243-4980-a8e9-9996e045add1 admin admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:07.086496 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb72994c-7243-4980-a8e9-9996e045add1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:07.088443 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb72994c-7243-4980-a8e9-9996e045add1 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:07.194805 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1375/2767] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:32:07 2022] POST /identity/v3/users => generated 286 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:32:07.198826 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04407af-2ed7-4fcd-9264-69e62b7661b0 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:07.209273 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04407af-2ed7-4fcd-9264-69e62b7661b0 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:07.213772 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686a2c12-2b47-43ef-954b-c6aedd49786a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:07.214364 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686a2c12-2b47-43ef-954b-c6aedd49786a admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:07.214643 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686a2c12-2b47-43ef-954b-c6aedd49786a admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:07.214936 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686a2c12-2b47-43ef-954b-c6aedd49786a admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:07.215571 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686a2c12-2b47-43ef-954b-c6aedd49786a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:07.217153 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686a2c12-2b47-43ef-954b-c6aedd49786a admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:07.226915 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1393/2768] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:32:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:32:07.230740 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb72994c-7243-4980-a8e9-9996e045add1 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:07.241637 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb72994c-7243-4980-a8e9-9996e045add1 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:07.246310 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556cb08d-2cf5-4745-b207-b2b4ff8b0a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:07.246966 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556cb08d-2cf5-4745-b207-b2b4ff8b0a17 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:07.247255 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556cb08d-2cf5-4745-b207-b2b4ff8b0a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:07.247533 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556cb08d-2cf5-4745-b207-b2b4ff8b0a17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:07.248164 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556cb08d-2cf5-4745-b207-b2b4ff8b0a17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:07.249756 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556cb08d-2cf5-4745-b207-b2b4ff8b0a17 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:07.260709 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1376/2769] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:32:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:32:07.264762 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686a2c12-2b47-43ef-954b-c6aedd49786a admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:07.274636 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686a2c12-2b47-43ef-954b-c6aedd49786a admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:07.279266 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685f2e64-a9ec-4781-96fe-815156406a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:07.279909 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f2e64-a9ec-4781-96fe-815156406a6f admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:07.280195 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f2e64-a9ec-4781-96fe-815156406a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:07.280474 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f2e64-a9ec-4781-96fe-815156406a6f admin admin] PATH_INFO: `/v3/projects/bf774e018bb044c9abff5a57e9d1996f/users/7923b451ff30497f8b7714771a3dbae5/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:07.294852 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685f2e64-a9ec-4781-96fe-815156406a6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf774e018bb044c9abff5a57e9d1996f, user_id=7923b451ff30497f8b7714771a3dbae5, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:07.297600 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685f2e64-a9ec-4781-96fe-815156406a6f admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:07.366505 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1394/2770] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:32:07 2022] PUT /identity/v3/projects/bf774e018bb044c9abff5a57e9d1996f/users/7923b451ff30497f8b7714771a3dbae5/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:32:07.377628 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a8b58-670a-4a31-a77d-98463e8f9071 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:07.377918 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a8b58-670a-4a31-a77d-98463e8f9071 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:07.378192 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a8b58-670a-4a31-a77d-98463e8f9071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:07.399009 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:32:07.407862 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-619a8b58-670a-4a31-a77d-98463e8f9071 None None] MFA Rules not processed for user `7923b451ff30497f8b7714771a3dbae5`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:32:07.409047 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-619a8b58-670a-4a31-a77d-98463e8f9071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:32:07.594014 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1377/2771] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:32:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:32:07.599437 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685f2e64-a9ec-4781-96fe-815156406a6f admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:07.612458 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:32:07.659581 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685f2e64-a9ec-4781-96fe-815156406a6f admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:07.664950 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2ddcbc-32d8-4f91-a0ef-3a1f8f46ee60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:07.665549 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2ddcbc-32d8-4f91-a0ef-3a1f8f46ee60 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:07.665827 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2ddcbc-32d8-4f91-a0ef-3a1f8f46ee60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:07.666094 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2ddcbc-32d8-4f91-a0ef-3a1f8f46ee60 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:07.666701 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2ddcbc-32d8-4f91-a0ef-3a1f8f46ee60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:07.668621 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2ddcbc-32d8-4f91-a0ef-3a1f8f46ee60 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:07.775620 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1395/2772] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:32:07 2022] POST /identity/v3/projects => generated 371 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:32:07.779634 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619a8b58-670a-4a31-a77d-98463e8f9071 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:07.790964 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619a8b58-670a-4a31-a77d-98463e8f9071 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:07.844494 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2245fa9-8365-43ba-b036-b8e561a1a3dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:07.845140 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2245fa9-8365-43ba-b036-b8e561a1a3dc admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:07.845339 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2245fa9-8365-43ba-b036-b8e561a1a3dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:07.845559 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2245fa9-8365-43ba-b036-b8e561a1a3dc admin admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:07.846166 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2245fa9-8365-43ba-b036-b8e561a1a3dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:07.848119 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2245fa9-8365-43ba-b036-b8e561a1a3dc admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:07.939761 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1378/2773] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:32:07 2022] POST /identity/v3/users => generated 292 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:32:07.944258 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2ddcbc-32d8-4f91-a0ef-3a1f8f46ee60 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:07.954680 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2ddcbc-32d8-4f91-a0ef-3a1f8f46ee60 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:07.960640 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7492cd95-e692-48e1-a350-b5a2cdaae6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:07.962609 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492cd95-e692-48e1-a350-b5a2cdaae6bd admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:07.962609 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492cd95-e692-48e1-a350-b5a2cdaae6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:07.962609 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492cd95-e692-48e1-a350-b5a2cdaae6bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:07.962609 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7492cd95-e692-48e1-a350-b5a2cdaae6bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:07.964193 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7492cd95-e692-48e1-a350-b5a2cdaae6bd admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:07.975726 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1396/2774] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:32:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:32:07.979950 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2245fa9-8365-43ba-b036-b8e561a1a3dc admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:07.989898 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2245fa9-8365-43ba-b036-b8e561a1a3dc admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:07.996640 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fc9065-0795-4340-8e59-cc35470dfa21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:07.996640 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc9065-0795-4340-8e59-cc35470dfa21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:07.996640 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc9065-0795-4340-8e59-cc35470dfa21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:07.996640 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc9065-0795-4340-8e59-cc35470dfa21 admin admin] PATH_INFO: `/v3/projects/bb312462e899476298e2663d08e486d7/users/3bf00cd2b6c846ff9b7618dda39d813e/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:08.010434 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fc9065-0795-4340-8e59-cc35470dfa21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb312462e899476298e2663d08e486d7, user_id=3bf00cd2b6c846ff9b7618dda39d813e, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:08.013119 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fc9065-0795-4340-8e59-cc35470dfa21 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:08.122020 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1379/2775] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:32:07 2022] PUT /identity/v3/projects/bb312462e899476298e2663d08e486d7/users/3bf00cd2b6c846ff9b7618dda39d813e/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:32:08.126204 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7492cd95-e692-48e1-a350-b5a2cdaae6bd admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:08.136768 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7492cd95-e692-48e1-a350-b5a2cdaae6bd admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:08.192018 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d890368d-cada-425b-85a0-3c322d7bc6ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:08.192624 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d890368d-cada-425b-85a0-3c322d7bc6ce admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:08.192910 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d890368d-cada-425b-85a0-3c322d7bc6ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:08.193753 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d890368d-cada-425b-85a0-3c322d7bc6ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:08.194422 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d890368d-cada-425b-85a0-3c322d7bc6ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:08.196056 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d890368d-cada-425b-85a0-3c322d7bc6ce admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:08.205714 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1397/2776] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:32:08 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:32:08.210715 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fc9065-0795-4340-8e59-cc35470dfa21 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:08.221035 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fc9065-0795-4340-8e59-cc35470dfa21 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:08.225579 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc754b56-3b81-46f0-a549-6af9e34afa41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:08.226223 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc754b56-3b81-46f0-a549-6af9e34afa41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:08.226506 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc754b56-3b81-46f0-a549-6af9e34afa41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:08.226783 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc754b56-3b81-46f0-a549-6af9e34afa41 admin admin] PATH_INFO: `/v3/projects/bb312462e899476298e2663d08e486d7/users/3bf00cd2b6c846ff9b7618dda39d813e/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:08.228853 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc754b56-3b81-46f0-a549-6af9e34afa41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb312462e899476298e2663d08e486d7, user_id=3bf00cd2b6c846ff9b7618dda39d813e, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:08.231507 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc754b56-3b81-46f0-a549-6af9e34afa41 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:08.242208 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1380/2777] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:32:08 2022] PUT /identity/v3/projects/bb312462e899476298e2663d08e486d7/users/3bf00cd2b6c846ff9b7618dda39d813e/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:32:08.252559 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eabafa8-b502-4361-870f-088e3f6566f4 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:08.252849 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eabafa8-b502-4361-870f-088e3f6566f4 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:08.261699 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eabafa8-b502-4361-870f-088e3f6566f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:08.282201 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-4eabafa8-b502-4361-870f-088e3f6566f4 None None] MFA Rules not processed for user `3bf00cd2b6c846ff9b7618dda39d813e`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:32:08.283443 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-4eabafa8-b502-4361-870f-088e3f6566f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:32:08.350685 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1398/2778] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:32:08 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:32:08.362204 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc754b56-3b81-46f0-a549-6af9e34afa41 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:08.363923 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-dc754b56-3b81-46f0-a549-6af9e34afa41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:32:08.419734 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc754b56-3b81-46f0-a549-6af9e34afa41 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:08.423269 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11a18ff-4e4d-4dc9-8efd-59b5f7ba3460 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:08.423897 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a18ff-4e4d-4dc9-8efd-59b5f7ba3460 service cinder] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:08.424177 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a18ff-4e4d-4dc9-8efd-59b5f7ba3460 service cinder] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:08.424451 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a18ff-4e4d-4dc9-8efd-59b5f7ba3460 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:08.431070 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:32:08.434626 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11a18ff-4e4d-4dc9-8efd-59b5f7ba3460 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:08.436130 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11a18ff-4e4d-4dc9-8efd-59b5f7ba3460 service cinder] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:08.446225 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1381/2779] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:32:08 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:32:09.333281 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eabafa8-b502-4361-870f-088e3f6566f4 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:09.343630 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:32:09.395228 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eabafa8-b502-4361-870f-088e3f6566f4 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:09.400368 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcfd0d8-5586-4a8d-92de-f14ce4a1b46f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:09.400989 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcfd0d8-5586-4a8d-92de-f14ce4a1b46f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:09.401281 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcfd0d8-5586-4a8d-92de-f14ce4a1b46f admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:09.401555 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcfd0d8-5586-4a8d-92de-f14ce4a1b46f admin admin] PATH_INFO: `/v3/users/7923b451ff30497f8b7714771a3dbae5` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:09.402666 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcfd0d8-5586-4a8d-92de-f14ce4a1b46f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7923b451ff30497f8b7714771a3dbae5)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:09.404675 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcfd0d8-5586-4a8d-92de-f14ce4a1b46f admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:09.580694 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-4fcfd0d8-5586-4a8d-92de-f14ce4a1b46f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7923b451ff30497f8b7714771a3dbae5'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:09.641958 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-4fcfd0d8-5586-4a8d-92de-f14ce4a1b46f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7923b451ff30497f8b7714771a3dbae5'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:09.642582 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-4fcfd0d8-5586-4a8d-92de-f14ce4a1b46f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7923b451ff30497f8b7714771a3dbae5'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:09.643203 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-4fcfd0d8-5586-4a8d-92de-f14ce4a1b46f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7923b451ff30497f8b7714771a3dbae5'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:09.651011 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-4fcfd0d8-5586-4a8d-92de-f14ce4a1b46f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7923b451ff30497f8b7714771a3dbae5'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:09.668297 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1399/2780] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:32:09 2022] DELETE /identity/v3/users/7923b451ff30497f8b7714771a3dbae5 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:32:09.672944 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a988c1-fbf1-481b-9f3b-9bf834805c46 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:09.673433 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a988c1-fbf1-481b-9f3b-9bf834805c46 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:09.673660 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a988c1-fbf1-481b-9f3b-9bf834805c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:09.691861 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-79a988c1-fbf1-481b-9f3b-9bf834805c46 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:32:09.693567 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-79a988c1-fbf1-481b-9f3b-9bf834805c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:32:09.749595 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1382/2781] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:32:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:32:09.759765 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcfd0d8-5586-4a8d-92de-f14ce4a1b46f admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:09.765944 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-4fcfd0d8-5586-4a8d-92de-f14ce4a1b46f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:32:09.819412 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcfd0d8-5586-4a8d-92de-f14ce4a1b46f admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:09.823003 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801a40fc-ef69-4c30-83ac-13fcec887227 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:09.823606 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801a40fc-ef69-4c30-83ac-13fcec887227 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:09.823883 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801a40fc-ef69-4c30-83ac-13fcec887227 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:09.824156 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801a40fc-ef69-4c30-83ac-13fcec887227 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:09.833141 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801a40fc-ef69-4c30-83ac-13fcec887227 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:09.835077 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801a40fc-ef69-4c30-83ac-13fcec887227 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:09.844548 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1400/2782] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:32:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:32:10.237686 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a988c1-fbf1-481b-9f3b-9bf834805c46 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:10.239173 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-79a988c1-fbf1-481b-9f3b-9bf834805c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:32:10.254257 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a988c1-fbf1-481b-9f3b-9bf834805c46 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:10.259873 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc767ea-23f2-4e98-9096-49d6d81e9c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:10.260509 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc767ea-23f2-4e98-9096-49d6d81e9c8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:10.260788 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc767ea-23f2-4e98-9096-49d6d81e9c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:10.261062 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc767ea-23f2-4e98-9096-49d6d81e9c8d admin admin] PATH_INFO: `/v3/projects/bf774e018bb044c9abff5a57e9d1996f` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:10.262470 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc767ea-23f2-4e98-9096-49d6d81e9c8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf774e018bb044c9abff5a57e9d1996f)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:10.264463 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc767ea-23f2-4e98-9096-49d6d81e9c8d admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:10.363124 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-ebc767ea-23f2-4e98-9096-49d6d81e9c8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf774e018bb044c9abff5a57e9d1996f', 'request_id': 'req-ebc767ea-23f2-4e98-9096-49d6d81e9c8d'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:10.436243 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-ebc767ea-23f2-4e98-9096-49d6d81e9c8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf774e018bb044c9abff5a57e9d1996f', 'request_id': 'req-ebc767ea-23f2-4e98-9096-49d6d81e9c8d'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:10.442311 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-ebc767ea-23f2-4e98-9096-49d6d81e9c8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf774e018bb044c9abff5a57e9d1996f', 'request_id': 'req-ebc767ea-23f2-4e98-9096-49d6d81e9c8d'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:10.454115 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-ebc767ea-23f2-4e98-9096-49d6d81e9c8d admin admin] The token cache is being invalidate because project bf774e018bb044c9abff5a57e9d1996f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81650) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:32:10.457556 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-ebc767ea-23f2-4e98-9096-49d6d81e9c8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:10.457556 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-ebc767ea-23f2-4e98-9096-49d6d81e9c8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:10.457556 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1383/2783] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:32:10 2022] DELETE /identity/v3/projects/bf774e018bb044c9abff5a57e9d1996f => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:32:10.460815 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801a40fc-ef69-4c30-83ac-13fcec887227 service neutron] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:10.470575 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801a40fc-ef69-4c30-83ac-13fcec887227 service neutron] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:10.500923 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:32:10.529871 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d7d0a6-d02b-4f5c-9c36-351eda404e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:10.530595 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d7d0a6-d02b-4f5c-9c36-351eda404e16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:10.530878 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d7d0a6-d02b-4f5c-9c36-351eda404e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:10.531179 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d7d0a6-d02b-4f5c-9c36-351eda404e16 admin admin] PATH_INFO: `/v3/users/3bf00cd2b6c846ff9b7618dda39d813e` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:10.533411 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d7d0a6-d02b-4f5c-9c36-351eda404e16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bf00cd2b6c846ff9b7618dda39d813e)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:10.535365 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d7d0a6-d02b-4f5c-9c36-351eda404e16 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:10.700817 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e5d7d0a6-d02b-4f5c-9c36-351eda404e16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bf00cd2b6c846ff9b7618dda39d813e'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:10.778983 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e5d7d0a6-d02b-4f5c-9c36-351eda404e16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bf00cd2b6c846ff9b7618dda39d813e'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:10.779605 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e5d7d0a6-d02b-4f5c-9c36-351eda404e16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bf00cd2b6c846ff9b7618dda39d813e'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:10.780226 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e5d7d0a6-d02b-4f5c-9c36-351eda404e16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bf00cd2b6c846ff9b7618dda39d813e'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:10.788339 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e5d7d0a6-d02b-4f5c-9c36-351eda404e16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bf00cd2b6c846ff9b7618dda39d813e'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:10.805685 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1401/2784] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:32:10 2022] DELETE /identity/v3/users/3bf00cd2b6c846ff9b7618dda39d813e => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:32:11.186833 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc767ea-23f2-4e98-9096-49d6d81e9c8d admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:11.188424 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-ebc767ea-23f2-4e98-9096-49d6d81e9c8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:32:11.203350 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc767ea-23f2-4e98-9096-49d6d81e9c8d admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:11.262064 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d7196e-94ee-4118-8905-8468ff7b0da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:11.262660 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d7196e-94ee-4118-8905-8468ff7b0da5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:11.262970 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d7196e-94ee-4118-8905-8468ff7b0da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:11.270016 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d7196e-94ee-4118-8905-8468ff7b0da5 admin admin] PATH_INFO: `/v3/projects/bb312462e899476298e2663d08e486d7` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:11.270016 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d7196e-94ee-4118-8905-8468ff7b0da5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb312462e899476298e2663d08e486d7)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:11.270016 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d7196e-94ee-4118-8905-8468ff7b0da5 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:11.287076 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:32:11.351461 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-21d7196e-94ee-4118-8905-8468ff7b0da5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb312462e899476298e2663d08e486d7', 'request_id': 'req-21d7196e-94ee-4118-8905-8468ff7b0da5'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:11.441532 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-21d7196e-94ee-4118-8905-8468ff7b0da5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb312462e899476298e2663d08e486d7', 'request_id': 'req-21d7196e-94ee-4118-8905-8468ff7b0da5'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:11.447723 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-21d7196e-94ee-4118-8905-8468ff7b0da5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb312462e899476298e2663d08e486d7', 'request_id': 'req-21d7196e-94ee-4118-8905-8468ff7b0da5'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:11.459175 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-21d7196e-94ee-4118-8905-8468ff7b0da5 admin admin] The token cache is being invalidate because project bb312462e899476298e2663d08e486d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81650) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:32:11.459990 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-21d7196e-94ee-4118-8905-8468ff7b0da5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:11.460903 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-21d7196e-94ee-4118-8905-8468ff7b0da5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:11.462367 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1384/2785] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:32:11 2022] DELETE /identity/v3/projects/bb312462e899476298e2663d08e486d7 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:32:11.467292 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4551046-7238-4961-b0c2-035acbef0250 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:11.467783 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4551046-7238-4961-b0c2-035acbef0250 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:11.467894 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4551046-7238-4961-b0c2-035acbef0250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:11.487939 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-d4551046-7238-4961-b0c2-035acbef0250 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:32:11.489131 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-d4551046-7238-4961-b0c2-035acbef0250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:32:11.546852 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1402/2786] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:32:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:32:11.556392 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a303d17-6e1c-4f66-8abf-940701256f41 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:11.556688 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a303d17-6e1c-4f66-8abf-940701256f41 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:11.556993 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a303d17-6e1c-4f66-8abf-940701256f41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:11.576810 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-7a303d17-6e1c-4f66-8abf-940701256f41 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:32:11.578007 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-7a303d17-6e1c-4f66-8abf-940701256f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:32:11.586635 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1385/2787] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:32:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:32:11.590790 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4551046-7238-4961-b0c2-035acbef0250 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:11.601194 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4551046-7238-4961-b0c2-035acbef0250 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:11.605728 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322d9f7d-d77b-436b-a15f-a776cbfef590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:11.606330 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d9f7d-d77b-436b-a15f-a776cbfef590 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:11.606598 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d9f7d-d77b-436b-a15f-a776cbfef590 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:11.606871 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d9f7d-d77b-436b-a15f-a776cbfef590 admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:11.607536 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322d9f7d-d77b-436b-a15f-a776cbfef590 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:32:11.607834 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322d9f7d-d77b-436b-a15f-a776cbfef590 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:11.609407 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322d9f7d-d77b-436b-a15f-a776cbfef590 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:11.620773 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1403/2788] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:32:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 17:32:11.624872 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a303d17-6e1c-4f66-8abf-940701256f41 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:11.634146 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a303d17-6e1c-4f66-8abf-940701256f41 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:11.638924 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3973945a-add7-44c3-b9ee-501b438da184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:11.639562 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3973945a-add7-44c3-b9ee-501b438da184 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:11.639841 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3973945a-add7-44c3-b9ee-501b438da184 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:11.640108 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3973945a-add7-44c3-b9ee-501b438da184 admin admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:11.640724 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3973945a-add7-44c3-b9ee-501b438da184 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:11.642616 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3973945a-add7-44c3-b9ee-501b438da184 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:11.746094 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1386/2789] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:32:11 2022] POST /identity/v3/projects => generated 369 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 17:32:11.750190 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322d9f7d-d77b-436b-a15f-a776cbfef590 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:11.760256 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322d9f7d-d77b-436b-a15f-a776cbfef590 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:11.794930 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:32:11.818143 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff4d777-e9ef-439e-9099-2456d7fd29c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:11.818681 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff4d777-e9ef-439e-9099-2456d7fd29c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:11.819601 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff4d777-e9ef-439e-9099-2456d7fd29c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:11.819882 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff4d777-e9ef-439e-9099-2456d7fd29c8 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:11.820512 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff4d777-e9ef-439e-9099-2456d7fd29c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:11.822446 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff4d777-e9ef-439e-9099-2456d7fd29c8 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:11.899199 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1404/2790] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:32:11 2022] POST /identity/v3/users => generated 285 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:32:11.903371 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3973945a-add7-44c3-b9ee-501b438da184 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:11.914201 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3973945a-add7-44c3-b9ee-501b438da184 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:11.918781 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42d8952-5bc6-4ee5-9655-e15ef5c4de45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:11.919411 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42d8952-5bc6-4ee5-9655-e15ef5c4de45 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:11.919693 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42d8952-5bc6-4ee5-9655-e15ef5c4de45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:11.919972 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42d8952-5bc6-4ee5-9655-e15ef5c4de45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:11.920607 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42d8952-5bc6-4ee5-9655-e15ef5c4de45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:11.922199 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42d8952-5bc6-4ee5-9655-e15ef5c4de45 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:11.933331 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1387/2791] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:32:11.937173 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff4d777-e9ef-439e-9099-2456d7fd29c8 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:11.947205 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff4d777-e9ef-439e-9099-2456d7fd29c8 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:11.951890 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150930ce-3d79-415b-a4ab-379813b8bbad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:11.952486 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150930ce-3d79-415b-a4ab-379813b8bbad admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:11.952768 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150930ce-3d79-415b-a4ab-379813b8bbad admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:11.953042 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150930ce-3d79-415b-a4ab-379813b8bbad admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:11.953680 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150930ce-3d79-415b-a4ab-379813b8bbad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:11.955431 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150930ce-3d79-415b-a4ab-379813b8bbad admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:11.965238 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1405/2792] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:32:11.970516 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42d8952-5bc6-4ee5-9655-e15ef5c4de45 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:11.980826 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42d8952-5bc6-4ee5-9655-e15ef5c4de45 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:11.985452 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f636cc6-fb95-46df-b33a-ad00848c3941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:11.986090 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f636cc6-fb95-46df-b33a-ad00848c3941 admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:11.986373 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f636cc6-fb95-46df-b33a-ad00848c3941 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:11.986646 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f636cc6-fb95-46df-b33a-ad00848c3941 admin admin] PATH_INFO: `/v3/projects/3e1ccf3ee43742f099b60c1279a8b6b3/users/3df511bfcf704766a96df2327f50055d/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:12.001617 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f636cc6-fb95-46df-b33a-ad00848c3941 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e1ccf3ee43742f099b60c1279a8b6b3, user_id=3df511bfcf704766a96df2327f50055d, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:12.004229 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f636cc6-fb95-46df-b33a-ad00848c3941 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:12.059565 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1388/2793] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:32:11 2022] PUT /identity/v3/projects/3e1ccf3ee43742f099b60c1279a8b6b3/users/3df511bfcf704766a96df2327f50055d/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:32:12.069957 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7da0ff-9226-4c7d-a31f-28e099885715 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:12.070213 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7da0ff-9226-4c7d-a31f-28e099885715 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:12.070449 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7da0ff-9226-4c7d-a31f-28e099885715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:12.097856 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-4b7da0ff-9226-4c7d-a31f-28e099885715 None None] MFA Rules not processed for user `3df511bfcf704766a96df2327f50055d`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:32:12.099125 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-4b7da0ff-9226-4c7d-a31f-28e099885715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:32:12.162454 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1406/2794] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:32:12 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:32:12.167814 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f636cc6-fb95-46df-b33a-ad00848c3941 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:12.229588 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f636cc6-fb95-46df-b33a-ad00848c3941 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:12.234184 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c0d7c0-5bc5-4aa4-9924-5c6545886580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:12.234781 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c0d7c0-5bc5-4aa4-9924-5c6545886580 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:12.235090 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c0d7c0-5bc5-4aa4-9924-5c6545886580 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:12.235363 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c0d7c0-5bc5-4aa4-9924-5c6545886580 admin admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:12.235979 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c0d7c0-5bc5-4aa4-9924-5c6545886580 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:12.237879 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c0d7c0-5bc5-4aa4-9924-5c6545886580 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:12.306487 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1389/2795] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:32:12 2022] POST /identity/v3/projects => generated 369 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 17:32:12.310690 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7da0ff-9226-4c7d-a31f-28e099885715 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:12.321831 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7da0ff-9226-4c7d-a31f-28e099885715 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:12.377612 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c2ccf6-5d9c-4e52-9838-949a91f952db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:12.378220 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c2ccf6-5d9c-4e52-9838-949a91f952db admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:12.378496 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c2ccf6-5d9c-4e52-9838-949a91f952db admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:12.378766 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c2ccf6-5d9c-4e52-9838-949a91f952db admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:12.379408 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c2ccf6-5d9c-4e52-9838-949a91f952db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:12.381314 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c2ccf6-5d9c-4e52-9838-949a91f952db admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:12.435938 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1407/2796] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:32:12 2022] POST /identity/v3/users => generated 291 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:32:12.440061 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c0d7c0-5bc5-4aa4-9924-5c6545886580 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:12.450757 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c0d7c0-5bc5-4aa4-9924-5c6545886580 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:12.455312 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98cca82-4476-4528-a389-63386fb8cc7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:12.455899 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98cca82-4476-4528-a389-63386fb8cc7f admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:12.456176 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98cca82-4476-4528-a389-63386fb8cc7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:12.456490 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98cca82-4476-4528-a389-63386fb8cc7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:12.457110 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98cca82-4476-4528-a389-63386fb8cc7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:12.458700 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98cca82-4476-4528-a389-63386fb8cc7f admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:12.470217 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1390/2797] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:32:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:32:12.474281 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c2ccf6-5d9c-4e52-9838-949a91f952db admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:12.484960 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c2ccf6-5d9c-4e52-9838-949a91f952db admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:12.489670 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6e5fe1-19b0-4429-abfe-6ef5c9edbd76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:12.490323 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e5fe1-19b0-4429-abfe-6ef5c9edbd76 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:12.490608 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e5fe1-19b0-4429-abfe-6ef5c9edbd76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:12.490887 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e5fe1-19b0-4429-abfe-6ef5c9edbd76 admin admin] PATH_INFO: `/v3/projects/0f86b045cd7c4bbaa221405ded92c41c/users/780b9ba157684a98ba82a6d7564c78ea/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:12.505322 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6e5fe1-19b0-4429-abfe-6ef5c9edbd76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f86b045cd7c4bbaa221405ded92c41c, user_id=780b9ba157684a98ba82a6d7564c78ea, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:12.508179 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6e5fe1-19b0-4429-abfe-6ef5c9edbd76 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:12.567701 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1408/2798] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:32:12 2022] PUT /identity/v3/projects/0f86b045cd7c4bbaa221405ded92c41c/users/780b9ba157684a98ba82a6d7564c78ea/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:32:12.572211 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98cca82-4476-4528-a389-63386fb8cc7f admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:12.582447 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98cca82-4476-4528-a389-63386fb8cc7f admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:12.595513 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:32:12.639109 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b3cae9-df6f-4a9a-9e01-a4f182c19fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:12.640024 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b3cae9-df6f-4a9a-9e01-a4f182c19fd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:12.640310 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b3cae9-df6f-4a9a-9e01-a4f182c19fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:12.640754 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b3cae9-df6f-4a9a-9e01-a4f182c19fd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:12.641756 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b3cae9-df6f-4a9a-9e01-a4f182c19fd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:12.643446 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b3cae9-df6f-4a9a-9e01-a4f182c19fd8 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:12.653885 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1391/2799] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:32:12 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:32:12.658132 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6e5fe1-19b0-4429-abfe-6ef5c9edbd76 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:12.668643 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6e5fe1-19b0-4429-abfe-6ef5c9edbd76 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:12.673482 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38db8997-52a8-4df2-a632-531c12c433af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:12.674142 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db8997-52a8-4df2-a632-531c12c433af admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:12.674420 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db8997-52a8-4df2-a632-531c12c433af admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:12.674693 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db8997-52a8-4df2-a632-531c12c433af admin admin] PATH_INFO: `/v3/projects/0f86b045cd7c4bbaa221405ded92c41c/users/780b9ba157684a98ba82a6d7564c78ea/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:12.676911 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38db8997-52a8-4df2-a632-531c12c433af admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f86b045cd7c4bbaa221405ded92c41c, user_id=780b9ba157684a98ba82a6d7564c78ea, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:12.679738 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38db8997-52a8-4df2-a632-531c12c433af admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:12.685273 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:32:12.690349 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1409/2800] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:32:12 2022] PUT /identity/v3/projects/0f86b045cd7c4bbaa221405ded92c41c/users/780b9ba157684a98ba82a6d7564c78ea/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:32:12.701377 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae3ad0-d6cd-4bbb-b9a8-5e45244b0a7e None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:12.701668 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae3ad0-d6cd-4bbb-b9a8-5e45244b0a7e None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:12.701983 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae3ad0-d6cd-4bbb-b9a8-5e45244b0a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:12.729206 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-f9ae3ad0-d6cd-4bbb-b9a8-5e45244b0a7e None None] MFA Rules not processed for user `780b9ba157684a98ba82a6d7564c78ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:32:12.730423 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-f9ae3ad0-d6cd-4bbb-b9a8-5e45244b0a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:32:12.797030 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1392/2801] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:32:12 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:32:12.808136 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38db8997-52a8-4df2-a632-531c12c433af admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:12.809850 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-38db8997-52a8-4df2-a632-531c12c433af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:32:12.868803 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38db8997-52a8-4df2-a632-531c12c433af admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:12.872335 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9d3894-48a2-4732-bef7-2bbd4741c7ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:12.872938 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d3894-48a2-4732-bef7-2bbd4741c7ec service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:12.873214 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d3894-48a2-4732-bef7-2bbd4741c7ec service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:12.873479 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d3894-48a2-4732-bef7-2bbd4741c7ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:12.882806 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9d3894-48a2-4732-bef7-2bbd4741c7ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:12.884463 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9d3894-48a2-4732-bef7-2bbd4741c7ec service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:12.893629 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1410/2802] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:32:12 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:32:15.675470 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ae3ad0-d6cd-4bbb-b9a8-5e45244b0a7e None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:15.736799 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ae3ad0-d6cd-4bbb-b9a8-5e45244b0a7e None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:15.741233 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eef587-0815-4142-9893-36ebd4ab10e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:15.742522 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eef587-0815-4142-9893-36ebd4ab10e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:15.742803 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eef587-0815-4142-9893-36ebd4ab10e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:15.743169 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eef587-0815-4142-9893-36ebd4ab10e8 admin admin] PATH_INFO: `/v3/users/3df511bfcf704766a96df2327f50055d` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:15.744276 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eef587-0815-4142-9893-36ebd4ab10e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3df511bfcf704766a96df2327f50055d)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:15.746221 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eef587-0815-4142-9893-36ebd4ab10e8 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:15.935127 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:32:15.939945 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-d9eef587-0815-4142-9893-36ebd4ab10e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3df511bfcf704766a96df2327f50055d'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:15.951787 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-d9eef587-0815-4142-9893-36ebd4ab10e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3df511bfcf704766a96df2327f50055d'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:15.952563 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-d9eef587-0815-4142-9893-36ebd4ab10e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3df511bfcf704766a96df2327f50055d'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:16.005338 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-d9eef587-0815-4142-9893-36ebd4ab10e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3df511bfcf704766a96df2327f50055d'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:16.005960 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-d9eef587-0815-4142-9893-36ebd4ab10e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3df511bfcf704766a96df2327f50055d'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:16.019013 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1393/2803] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:32:15 2022] DELETE /identity/v3/users/3df511bfcf704766a96df2327f50055d => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:32:16.024106 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89291ebd-7897-49b8-a9f2-b10f2e0ca270 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:16.026828 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89291ebd-7897-49b8-a9f2-b10f2e0ca270 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:16.026828 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89291ebd-7897-49b8-a9f2-b10f2e0ca270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:16.043728 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-89291ebd-7897-49b8-a9f2-b10f2e0ca270 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:32:16.044920 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-89291ebd-7897-49b8-a9f2-b10f2e0ca270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:32:16.102828 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1411/2804] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:32:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:32:16.112309 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eef587-0815-4142-9893-36ebd4ab10e8 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:16.113756 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-d9eef587-0815-4142-9893-36ebd4ab10e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:32:16.170205 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eef587-0815-4142-9893-36ebd4ab10e8 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:16.173903 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13388883-13aa-48f9-8496-30c9f3a2c906 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:16.174534 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13388883-13aa-48f9-8496-30c9f3a2c906 service neutron] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:16.174815 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13388883-13aa-48f9-8496-30c9f3a2c906 service neutron] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:16.175123 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13388883-13aa-48f9-8496-30c9f3a2c906 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:16.184600 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13388883-13aa-48f9-8496-30c9f3a2c906 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:16.186086 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13388883-13aa-48f9-8496-30c9f3a2c906 service neutron] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:16.195804 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1394/2805] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:32:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:32:16.574683 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89291ebd-7897-49b8-a9f2-b10f2e0ca270 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:16.576259 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-89291ebd-7897-49b8-a9f2-b10f2e0ca270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:32:16.588750 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89291ebd-7897-49b8-a9f2-b10f2e0ca270 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:16.593690 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95e681f-dbea-4b85-be01-c17d93fc7826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:16.594372 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e681f-dbea-4b85-be01-c17d93fc7826 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:16.594660 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e681f-dbea-4b85-be01-c17d93fc7826 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:16.594957 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e681f-dbea-4b85-be01-c17d93fc7826 admin admin] PATH_INFO: `/v3/projects/3e1ccf3ee43742f099b60c1279a8b6b3` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:16.596207 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95e681f-dbea-4b85-be01-c17d93fc7826 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e1ccf3ee43742f099b60c1279a8b6b3)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:16.598490 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95e681f-dbea-4b85-be01-c17d93fc7826 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:16.691652 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e95e681f-dbea-4b85-be01-c17d93fc7826 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e1ccf3ee43742f099b60c1279a8b6b3', 'request_id': 'req-e95e681f-dbea-4b85-be01-c17d93fc7826'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:16.698355 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e95e681f-dbea-4b85-be01-c17d93fc7826 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e1ccf3ee43742f099b60c1279a8b6b3', 'request_id': 'req-e95e681f-dbea-4b85-be01-c17d93fc7826'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:16.771360 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e95e681f-dbea-4b85-be01-c17d93fc7826 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e1ccf3ee43742f099b60c1279a8b6b3', 'request_id': 'req-e95e681f-dbea-4b85-be01-c17d93fc7826'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:16.782717 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e95e681f-dbea-4b85-be01-c17d93fc7826 admin admin] The token cache is being invalidate because project 3e1ccf3ee43742f099b60c1279a8b6b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:32:16.783038 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e95e681f-dbea-4b85-be01-c17d93fc7826 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:16.783699 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e95e681f-dbea-4b85-be01-c17d93fc7826 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:16.785199 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1412/2806] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:32:16 2022] DELETE /identity/v3/projects/3e1ccf3ee43742f099b60c1279a8b6b3 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:32:16.789345 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13388883-13aa-48f9-8496-30c9f3a2c906 service neutron] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:16.799029 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13388883-13aa-48f9-8496-30c9f3a2c906 service neutron] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:16.856180 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c947209-0e0b-482a-8c2e-6c0ea39196ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:16.856802 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c947209-0e0b-482a-8c2e-6c0ea39196ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:16.857081 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c947209-0e0b-482a-8c2e-6c0ea39196ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:16.857351 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c947209-0e0b-482a-8c2e-6c0ea39196ec admin admin] PATH_INFO: `/v3/users/780b9ba157684a98ba82a6d7564c78ea` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:16.858476 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c947209-0e0b-482a-8c2e-6c0ea39196ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=780b9ba157684a98ba82a6d7564c78ea)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:16.860407 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c947209-0e0b-482a-8c2e-6c0ea39196ec admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:17.001871 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-6c947209-0e0b-482a-8c2e-6c0ea39196ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '780b9ba157684a98ba82a6d7564c78ea'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:17.013393 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:32:17.014537 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-6c947209-0e0b-482a-8c2e-6c0ea39196ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '780b9ba157684a98ba82a6d7564c78ea'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:17.015237 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-6c947209-0e0b-482a-8c2e-6c0ea39196ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '780b9ba157684a98ba82a6d7564c78ea'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:17.087122 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-6c947209-0e0b-482a-8c2e-6c0ea39196ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '780b9ba157684a98ba82a6d7564c78ea'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:17.087747 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-6c947209-0e0b-482a-8c2e-6c0ea39196ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '780b9ba157684a98ba82a6d7564c78ea'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:17.099803 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1395/2807] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:32:16 2022] DELETE /identity/v3/users/780b9ba157684a98ba82a6d7564c78ea => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:32:17.552006 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95e681f-dbea-4b85-be01-c17d93fc7826 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:17.553429 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-e95e681f-dbea-4b85-be01-c17d93fc7826 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:32:17.567267 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95e681f-dbea-4b85-be01-c17d93fc7826 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:17.588295 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:32:17.624593 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b738c2d-ca41-4185-ae6d-c1fcc6df62e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:17.625223 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b738c2d-ca41-4185-ae6d-c1fcc6df62e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:17.625507 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b738c2d-ca41-4185-ae6d-c1fcc6df62e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:17.625783 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b738c2d-ca41-4185-ae6d-c1fcc6df62e2 admin admin] PATH_INFO: `/v3/projects/0f86b045cd7c4bbaa221405ded92c41c` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:17.626944 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b738c2d-ca41-4185-ae6d-c1fcc6df62e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f86b045cd7c4bbaa221405ded92c41c)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:17.628850 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b738c2d-ca41-4185-ae6d-c1fcc6df62e2 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:17.730640 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-2b738c2d-ca41-4185-ae6d-c1fcc6df62e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f86b045cd7c4bbaa221405ded92c41c', 'request_id': 'req-2b738c2d-ca41-4185-ae6d-c1fcc6df62e2'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:17.736664 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-2b738c2d-ca41-4185-ae6d-c1fcc6df62e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f86b045cd7c4bbaa221405ded92c41c', 'request_id': 'req-2b738c2d-ca41-4185-ae6d-c1fcc6df62e2'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:17.799458 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-2b738c2d-ca41-4185-ae6d-c1fcc6df62e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f86b045cd7c4bbaa221405ded92c41c', 'request_id': 'req-2b738c2d-ca41-4185-ae6d-c1fcc6df62e2'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:17.811335 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-2b738c2d-ca41-4185-ae6d-c1fcc6df62e2 admin admin] The token cache is being invalidate because project 0f86b045cd7c4bbaa221405ded92c41c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:32:17.811619 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-2b738c2d-ca41-4185-ae6d-c1fcc6df62e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:17.812286 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-2b738c2d-ca41-4185-ae6d-c1fcc6df62e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:17.813812 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1413/2808] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:32:17 2022] DELETE /identity/v3/projects/0f86b045cd7c4bbaa221405ded92c41c => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:32:17.818822 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dc90a6-89d7-43ad-8b45-d345ec49c6e8 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:17.819305 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dc90a6-89d7-43ad-8b45-d345ec49c6e8 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:17.819404 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dc90a6-89d7-43ad-8b45-d345ec49c6e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:17.838364 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-19dc90a6-89d7-43ad-8b45-d345ec49c6e8 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:32:17.839579 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-19dc90a6-89d7-43ad-8b45-d345ec49c6e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:32:17.897167 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1396/2809] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:32:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:32:17.959774 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274fb0c5-e8d4-4e9f-b526-3402a9a1387c None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:17.960058 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274fb0c5-e8d4-4e9f-b526-3402a9a1387c None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:17.960441 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274fb0c5-e8d4-4e9f-b526-3402a9a1387c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:17.980807 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-274fb0c5-e8d4-4e9f-b526-3402a9a1387c None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:32:17.982075 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-274fb0c5-e8d4-4e9f-b526-3402a9a1387c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:32:17.989871 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1414/2810] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:32:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:32:17.994882 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dc90a6-89d7-43ad-8b45-d345ec49c6e8 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:18.005028 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dc90a6-89d7-43ad-8b45-d345ec49c6e8 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:18.009763 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b739b97f-58c1-4af5-b6e0-635da2c25db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:18.010365 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b739b97f-58c1-4af5-b6e0-635da2c25db9 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:18.010648 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b739b97f-58c1-4af5-b6e0-635da2c25db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:18.010948 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b739b97f-58c1-4af5-b6e0-635da2c25db9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:18.011585 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b739b97f-58c1-4af5-b6e0-635da2c25db9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:32:18.011911 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b739b97f-58c1-4af5-b6e0-635da2c25db9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:18.013511 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b739b97f-58c1-4af5-b6e0-635da2c25db9 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:18.025134 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1397/2811] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:32:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 17:32:18.029639 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274fb0c5-e8d4-4e9f-b526-3402a9a1387c None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:18.039742 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274fb0c5-e8d4-4e9f-b526-3402a9a1387c None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:18.044433 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829ebb90-f303-45ee-abca-3ebbf2dd6c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:18.045028 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829ebb90-f303-45ee-abca-3ebbf2dd6c34 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:18.045307 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829ebb90-f303-45ee-abca-3ebbf2dd6c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:18.045607 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829ebb90-f303-45ee-abca-3ebbf2dd6c34 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:18.046220 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829ebb90-f303-45ee-abca-3ebbf2dd6c34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:18.048212 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829ebb90-f303-45ee-abca-3ebbf2dd6c34 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:18.131839 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1415/2812] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:32:18 2022] POST /identity/v3/projects => generated 379 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:32:18.136167 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b739b97f-58c1-4af5-b6e0-635da2c25db9 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:18.146207 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b739b97f-58c1-4af5-b6e0-635da2c25db9 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:18.202161 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf2b3ea-e1d7-4043-97e1-856bdc8b6f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:18.202765 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2b3ea-e1d7-4043-97e1-856bdc8b6f98 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:18.203080 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2b3ea-e1d7-4043-97e1-856bdc8b6f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:18.203356 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2b3ea-e1d7-4043-97e1-856bdc8b6f98 admin admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:18.203982 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf2b3ea-e1d7-4043-97e1-856bdc8b6f98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:18.205942 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf2b3ea-e1d7-4043-97e1-856bdc8b6f98 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:18.302411 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1398/2813] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:32:18 2022] POST /identity/v3/users => generated 290 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:32:18.306806 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829ebb90-f303-45ee-abca-3ebbf2dd6c34 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:18.317336 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829ebb90-f303-45ee-abca-3ebbf2dd6c34 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:18.322203 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d6f276-9165-4b11-9a1a-cd04b9b8ddae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:18.322808 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d6f276-9165-4b11-9a1a-cd04b9b8ddae admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:18.323512 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d6f276-9165-4b11-9a1a-cd04b9b8ddae admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:18.323512 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d6f276-9165-4b11-9a1a-cd04b9b8ddae admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:18.324051 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d6f276-9165-4b11-9a1a-cd04b9b8ddae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:18.325711 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d6f276-9165-4b11-9a1a-cd04b9b8ddae admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:18.336169 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1416/2814] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:32:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:32:18.340191 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf2b3ea-e1d7-4043-97e1-856bdc8b6f98 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:18.350168 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf2b3ea-e1d7-4043-97e1-856bdc8b6f98 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:18.355518 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8cca4b-4ffe-402e-8117-7656c76344b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:18.356116 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8cca4b-4ffe-402e-8117-7656c76344b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:18.356398 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8cca4b-4ffe-402e-8117-7656c76344b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:18.356698 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8cca4b-4ffe-402e-8117-7656c76344b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:18.357328 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8cca4b-4ffe-402e-8117-7656c76344b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:18.358950 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8cca4b-4ffe-402e-8117-7656c76344b0 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:18.369172 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1399/2815] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:32:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:32:18.373207 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d6f276-9165-4b11-9a1a-cd04b9b8ddae admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:18.383053 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d6f276-9165-4b11-9a1a-cd04b9b8ddae admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:18.387485 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035d77cb-3e5f-4e03-9d2d-6bc0cf27cc37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:18.388123 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d77cb-3e5f-4e03-9d2d-6bc0cf27cc37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:18.388399 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d77cb-3e5f-4e03-9d2d-6bc0cf27cc37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:18.388667 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d77cb-3e5f-4e03-9d2d-6bc0cf27cc37 admin admin] PATH_INFO: `/v3/projects/8d65c244675d4dcf9208d1778a64df22/users/ee035af1d1b346b4a49e43f674c813c9/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:18.403225 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035d77cb-3e5f-4e03-9d2d-6bc0cf27cc37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d65c244675d4dcf9208d1778a64df22, user_id=ee035af1d1b346b4a49e43f674c813c9, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:18.405827 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035d77cb-3e5f-4e03-9d2d-6bc0cf27cc37 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:18.502514 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1417/2816] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:32:18 2022] PUT /identity/v3/projects/8d65c244675d4dcf9208d1778a64df22/users/ee035af1d1b346b4a49e43f674c813c9/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:32:18.512353 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b07944d-99e0-4250-a570-345ab8d3f865 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:18.512642 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b07944d-99e0-4250-a570-345ab8d3f865 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:18.512918 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b07944d-99e0-4250-a570-345ab8d3f865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:18.540620 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-0b07944d-99e0-4250-a570-345ab8d3f865 None None] MFA Rules not processed for user `ee035af1d1b346b4a49e43f674c813c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:32:18.541893 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-0b07944d-99e0-4250-a570-345ab8d3f865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:32:18.573995 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:32:18.603921 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1400/2817] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:32:18 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:32:18.608393 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035d77cb-3e5f-4e03-9d2d-6bc0cf27cc37 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:18.658849 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:32:18.669719 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035d77cb-3e5f-4e03-9d2d-6bc0cf27cc37 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:18.680737 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b1e4f-3fbb-49d6-a4a5-4eb1e1c01a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:18.680737 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b1e4f-3fbb-49d6-a4a5-4eb1e1c01a19 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:18.680737 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b1e4f-3fbb-49d6-a4a5-4eb1e1c01a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:18.680737 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b1e4f-3fbb-49d6-a4a5-4eb1e1c01a19 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:18.681147 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60b1e4f-3fbb-49d6-a4a5-4eb1e1c01a19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:18.681147 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60b1e4f-3fbb-49d6-a4a5-4eb1e1c01a19 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:18.756463 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1418/2818] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:32:18 2022] POST /identity/v3/projects => generated 383 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:32:18.761511 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b07944d-99e0-4250-a570-345ab8d3f865 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:18.772308 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b07944d-99e0-4250-a570-345ab8d3f865 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:18.827683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c372074a-4b0b-44cf-8ed4-4ab2ff01b349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:18.828370 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c372074a-4b0b-44cf-8ed4-4ab2ff01b349 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:18.828585 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c372074a-4b0b-44cf-8ed4-4ab2ff01b349 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:18.828857 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c372074a-4b0b-44cf-8ed4-4ab2ff01b349 admin admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:18.829463 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c372074a-4b0b-44cf-8ed4-4ab2ff01b349 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:18.831415 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c372074a-4b0b-44cf-8ed4-4ab2ff01b349 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:18.927220 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1401/2819] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:32:18 2022] POST /identity/v3/users => generated 298 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:32:18.931673 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b1e4f-3fbb-49d6-a4a5-4eb1e1c01a19 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:18.941895 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b1e4f-3fbb-49d6-a4a5-4eb1e1c01a19 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:18.947251 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f569d3-3cb0-408d-ac2b-6a12e7d4d61b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:18.947854 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f569d3-3cb0-408d-ac2b-6a12e7d4d61b admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:18.948134 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f569d3-3cb0-408d-ac2b-6a12e7d4d61b admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:18.948405 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f569d3-3cb0-408d-ac2b-6a12e7d4d61b admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:18.950131 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f569d3-3cb0-408d-ac2b-6a12e7d4d61b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:18.950613 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f569d3-3cb0-408d-ac2b-6a12e7d4d61b admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:18.960637 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1419/2820] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:32:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:32:18.964633 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c372074a-4b0b-44cf-8ed4-4ab2ff01b349 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:18.974553 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c372074a-4b0b-44cf-8ed4-4ab2ff01b349 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:18.979146 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3744b6-a18a-4f5a-aaca-b57bb11b87a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:18.979784 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3744b6-a18a-4f5a-aaca-b57bb11b87a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:18.980064 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3744b6-a18a-4f5a-aaca-b57bb11b87a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:18.980352 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3744b6-a18a-4f5a-aaca-b57bb11b87a4 admin admin] PATH_INFO: `/v3/projects/8a993d01aa5e449d8ba953def105d648/users/e3677362866a4d10a5717956f1146d8c/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:18.995668 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3744b6-a18a-4f5a-aaca-b57bb11b87a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a993d01aa5e449d8ba953def105d648, user_id=e3677362866a4d10a5717956f1146d8c, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:18.998313 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3744b6-a18a-4f5a-aaca-b57bb11b87a4 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:19.062606 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1402/2821] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:32:18 2022] PUT /identity/v3/projects/8a993d01aa5e449d8ba953def105d648/users/e3677362866a4d10a5717956f1146d8c/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:32:19.066517 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f569d3-3cb0-408d-ac2b-6a12e7d4d61b admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:19.077188 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f569d3-3cb0-408d-ac2b-6a12e7d4d61b admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:19.131439 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b1f68e-3de9-44c1-8cd6-f3a34d1161af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:19.132098 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b1f68e-3de9-44c1-8cd6-f3a34d1161af admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:19.132381 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b1f68e-3de9-44c1-8cd6-f3a34d1161af admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:19.133144 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b1f68e-3de9-44c1-8cd6-f3a34d1161af admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:19.133763 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b1f68e-3de9-44c1-8cd6-f3a34d1161af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:19.135380 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b1f68e-3de9-44c1-8cd6-f3a34d1161af admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:19.144805 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1420/2822] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:32:19 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:32:19.148933 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3744b6-a18a-4f5a-aaca-b57bb11b87a4 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:19.159036 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3744b6-a18a-4f5a-aaca-b57bb11b87a4 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:19.163630 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64149bac-7a42-4918-ac83-39b2757d8ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:19.164262 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64149bac-7a42-4918-ac83-39b2757d8ff0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:19.164545 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64149bac-7a42-4918-ac83-39b2757d8ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:19.164823 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64149bac-7a42-4918-ac83-39b2757d8ff0 admin admin] PATH_INFO: `/v3/projects/8a993d01aa5e449d8ba953def105d648/users/e3677362866a4d10a5717956f1146d8c/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:19.166724 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64149bac-7a42-4918-ac83-39b2757d8ff0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a993d01aa5e449d8ba953def105d648, user_id=e3677362866a4d10a5717956f1146d8c, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:19.169378 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64149bac-7a42-4918-ac83-39b2757d8ff0 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:19.180350 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1403/2823] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:32:19 2022] PUT /identity/v3/projects/8a993d01aa5e449d8ba953def105d648/users/e3677362866a4d10a5717956f1146d8c/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:32:19.191106 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d77a21-eeca-4372-8b37-9c6efa3cb1eb None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:19.191398 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d77a21-eeca-4372-8b37-9c6efa3cb1eb None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:19.191706 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d77a21-eeca-4372-8b37-9c6efa3cb1eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:19.222794 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-16d77a21-eeca-4372-8b37-9c6efa3cb1eb None None] MFA Rules not processed for user `e3677362866a4d10a5717956f1146d8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:32:19.224031 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-16d77a21-eeca-4372-8b37-9c6efa3cb1eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:32:19.291054 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1421/2824] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:32:19 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:32:19.306396 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64149bac-7a42-4918-ac83-39b2757d8ff0 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:19.308129 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-64149bac-7a42-4918-ac83-39b2757d8ff0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:32:19.363965 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64149bac-7a42-4918-ac83-39b2757d8ff0 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:19.368400 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6728c6-15a8-4974-aed9-9b747c645579 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:19.369001 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6728c6-15a8-4974-aed9-9b747c645579 service cinder] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:19.369284 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6728c6-15a8-4974-aed9-9b747c645579 service cinder] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:19.369564 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6728c6-15a8-4974-aed9-9b747c645579 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:19.378417 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6728c6-15a8-4974-aed9-9b747c645579 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:19.379890 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6728c6-15a8-4974-aed9-9b747c645579 service cinder] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:19.389178 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1404/2825] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:32:19 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:32:19.545442 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d77a21-eeca-4372-8b37-9c6efa3cb1eb None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:19.556013 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d77a21-eeca-4372-8b37-9c6efa3cb1eb None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:19.559521 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fbf83c-7c80-4f46-a92a-2df171d4e2f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:19.560115 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fbf83c-7c80-4f46-a92a-2df171d4e2f4 service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:19.564672 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fbf83c-7c80-4f46-a92a-2df171d4e2f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:19.564672 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fbf83c-7c80-4f46-a92a-2df171d4e2f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:19.604432 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:32:19.617389 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fbf83c-7c80-4f46-a92a-2df171d4e2f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:19.618822 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fbf83c-7c80-4f46-a92a-2df171d4e2f4 service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:19.627680 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1422/2826] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:32:19 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:32:20.081972 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6728c6-15a8-4974-aed9-9b747c645579 service cinder] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:20.114657 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:32:20.142579 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6728c6-15a8-4974-aed9-9b747c645579 service cinder] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:20.147641 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99648d46-888b-4ed4-9c74-cbc130ce3507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:20.148261 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99648d46-888b-4ed4-9c74-cbc130ce3507 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:20.148538 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99648d46-888b-4ed4-9c74-cbc130ce3507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:20.148809 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99648d46-888b-4ed4-9c74-cbc130ce3507 admin admin] PATH_INFO: `/v3/users/ee035af1d1b346b4a49e43f674c813c9` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:20.149902 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99648d46-888b-4ed4-9c74-cbc130ce3507 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee035af1d1b346b4a49e43f674c813c9)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:20.151849 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99648d46-888b-4ed4-9c74-cbc130ce3507 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:20.304661 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-99648d46-888b-4ed4-9c74-cbc130ce3507 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee035af1d1b346b4a49e43f674c813c9'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:20.316523 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-99648d46-888b-4ed4-9c74-cbc130ce3507 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee035af1d1b346b4a49e43f674c813c9'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:20.317132 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-99648d46-888b-4ed4-9c74-cbc130ce3507 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee035af1d1b346b4a49e43f674c813c9'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:20.378663 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-99648d46-888b-4ed4-9c74-cbc130ce3507 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee035af1d1b346b4a49e43f674c813c9'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:20.379326 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-99648d46-888b-4ed4-9c74-cbc130ce3507 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee035af1d1b346b4a49e43f674c813c9'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:20.392469 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1405/2827] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:32:20 2022] DELETE /identity/v3/users/ee035af1d1b346b4a49e43f674c813c9 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:32:20.396928 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee1250f-6d7e-4dcb-9c42-4682e1b4bd2a None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:20.397218 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee1250f-6d7e-4dcb-9c42-4682e1b4bd2a None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:20.397497 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee1250f-6d7e-4dcb-9c42-4682e1b4bd2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:20.416477 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-fee1250f-6d7e-4dcb-9c42-4682e1b4bd2a None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:32:20.417668 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-fee1250f-6d7e-4dcb-9c42-4682e1b4bd2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:32:20.476524 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1423/2828] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:32:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:32:20.486315 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99648d46-888b-4ed4-9c74-cbc130ce3507 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:20.487758 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-99648d46-888b-4ed4-9c74-cbc130ce3507 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:32:20.544983 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99648d46-888b-4ed4-9c74-cbc130ce3507 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:20.548603 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b932af3d-215d-4ddd-99d0-7efc56e8ee7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:20.549205 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b932af3d-215d-4ddd-99d0-7efc56e8ee7e service neutron] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:20.549481 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b932af3d-215d-4ddd-99d0-7efc56e8ee7e service neutron] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:20.549771 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b932af3d-215d-4ddd-99d0-7efc56e8ee7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:20.559519 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b932af3d-215d-4ddd-99d0-7efc56e8ee7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:20.560979 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b932af3d-215d-4ddd-99d0-7efc56e8ee7e service neutron] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:20.570191 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1406/2829] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:32:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:32:20.986981 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee1250f-6d7e-4dcb-9c42-4682e1b4bd2a None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:20.988448 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-fee1250f-6d7e-4dcb-9c42-4682e1b4bd2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:32:21.001343 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee1250f-6d7e-4dcb-9c42-4682e1b4bd2a None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:21.006167 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a435dcfe-8810-45ed-ab15-da1045788c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:21.006811 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a435dcfe-8810-45ed-ab15-da1045788c85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:21.007141 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a435dcfe-8810-45ed-ab15-da1045788c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:21.007418 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a435dcfe-8810-45ed-ab15-da1045788c85 admin admin] PATH_INFO: `/v3/projects/8d65c244675d4dcf9208d1778a64df22` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:21.008544 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a435dcfe-8810-45ed-ab15-da1045788c85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d65c244675d4dcf9208d1778a64df22)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:21.011040 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a435dcfe-8810-45ed-ab15-da1045788c85 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:21.139046 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-a435dcfe-8810-45ed-ab15-da1045788c85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d65c244675d4dcf9208d1778a64df22', 'request_id': 'req-a435dcfe-8810-45ed-ab15-da1045788c85'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:21.145390 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-a435dcfe-8810-45ed-ab15-da1045788c85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d65c244675d4dcf9208d1778a64df22', 'request_id': 'req-a435dcfe-8810-45ed-ab15-da1045788c85'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:21.233771 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-a435dcfe-8810-45ed-ab15-da1045788c85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d65c244675d4dcf9208d1778a64df22', 'request_id': 'req-a435dcfe-8810-45ed-ab15-da1045788c85'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:21.244475 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-a435dcfe-8810-45ed-ab15-da1045788c85 admin admin] The token cache is being invalidate because project 8d65c244675d4dcf9208d1778a64df22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:32:21.244756 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-a435dcfe-8810-45ed-ab15-da1045788c85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:21.245425 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-a435dcfe-8810-45ed-ab15-da1045788c85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:21.246970 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1424/2830] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:32:20 2022] DELETE /identity/v3/projects/8d65c244675d4dcf9208d1778a64df22 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:32:21.251094 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b932af3d-215d-4ddd-99d0-7efc56e8ee7e service neutron] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:21.261430 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b932af3d-215d-4ddd-99d0-7efc56e8ee7e service neutron] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:21.308526 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:32:21.317000 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a5b630-ca19-4ded-87a0-7c10f0cdf5e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:21.317623 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a5b630-ca19-4ded-87a0-7c10f0cdf5e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:21.317904 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a5b630-ca19-4ded-87a0-7c10f0cdf5e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:21.321077 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a5b630-ca19-4ded-87a0-7c10f0cdf5e7 admin admin] PATH_INFO: `/v3/users/e3677362866a4d10a5717956f1146d8c` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:21.321077 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a5b630-ca19-4ded-87a0-7c10f0cdf5e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3677362866a4d10a5717956f1146d8c)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:21.321393 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a5b630-ca19-4ded-87a0-7c10f0cdf5e7 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:21.552012 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-37a5b630-ca19-4ded-87a0-7c10f0cdf5e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3677362866a4d10a5717956f1146d8c'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:21.563692 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-37a5b630-ca19-4ded-87a0-7c10f0cdf5e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3677362866a4d10a5717956f1146d8c'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:21.564465 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-37a5b630-ca19-4ded-87a0-7c10f0cdf5e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3677362866a4d10a5717956f1146d8c'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:21.630108 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-37a5b630-ca19-4ded-87a0-7c10f0cdf5e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3677362866a4d10a5717956f1146d8c'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:21.630773 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-37a5b630-ca19-4ded-87a0-7c10f0cdf5e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3677362866a4d10a5717956f1146d8c'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:21.643746 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1407/2831] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:32:21 2022] DELETE /identity/v3/users/e3677362866a4d10a5717956f1146d8c => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:32:22.076206 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a435dcfe-8810-45ed-ab15-da1045788c85 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:22.077657 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-a435dcfe-8810-45ed-ab15-da1045788c85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:32:22.094663 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a435dcfe-8810-45ed-ab15-da1045788c85 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:22.153562 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e9519a-bcb2-43b0-ba17-1916b20af8ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:22.154197 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e9519a-bcb2-43b0-ba17-1916b20af8ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:22.154535 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e9519a-bcb2-43b0-ba17-1916b20af8ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:22.154816 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e9519a-bcb2-43b0-ba17-1916b20af8ef admin admin] PATH_INFO: `/v3/projects/8a993d01aa5e449d8ba953def105d648` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:22.156005 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e9519a-bcb2-43b0-ba17-1916b20af8ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a993d01aa5e449d8ba953def105d648)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:22.157913 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e9519a-bcb2-43b0-ba17-1916b20af8ef admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:22.168746 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:32:22.276560 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-91e9519a-bcb2-43b0-ba17-1916b20af8ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a993d01aa5e449d8ba953def105d648', 'request_id': 'req-91e9519a-bcb2-43b0-ba17-1916b20af8ef'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:22.282831 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-91e9519a-bcb2-43b0-ba17-1916b20af8ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a993d01aa5e449d8ba953def105d648', 'request_id': 'req-91e9519a-bcb2-43b0-ba17-1916b20af8ef'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:22.370601 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-91e9519a-bcb2-43b0-ba17-1916b20af8ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a993d01aa5e449d8ba953def105d648', 'request_id': 'req-91e9519a-bcb2-43b0-ba17-1916b20af8ef'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:22.381686 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-91e9519a-bcb2-43b0-ba17-1916b20af8ef admin admin] The token cache is being invalidate because project 8a993d01aa5e449d8ba953def105d648 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:32:22.382270 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-91e9519a-bcb2-43b0-ba17-1916b20af8ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:22.383000 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-91e9519a-bcb2-43b0-ba17-1916b20af8ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:22.384937 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1425/2832] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:32:22 2022] DELETE /identity/v3/projects/8a993d01aa5e449d8ba953def105d648 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:32:22.389847 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c447f013-7a04-4e16-895d-4e91d25e3a02 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:22.390314 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c447f013-7a04-4e16-895d-4e91d25e3a02 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:22.390414 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c447f013-7a04-4e16-895d-4e91d25e3a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:22.409493 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-c447f013-7a04-4e16-895d-4e91d25e3a02 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:32:22.410669 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-c447f013-7a04-4e16-895d-4e91d25e3a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:32:22.471942 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1408/2833] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:32:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:32:22.484010 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf35481-741e-4af6-b1b7-c9b728c58f0b None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:22.484295 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf35481-741e-4af6-b1b7-c9b728c58f0b None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:22.484574 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf35481-741e-4af6-b1b7-c9b728c58f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:22.504591 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-5bf35481-741e-4af6-b1b7-c9b728c58f0b None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:32:22.505788 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-5bf35481-741e-4af6-b1b7-c9b728c58f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:32:22.513710 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1426/2834] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:32:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:32:22.517821 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c447f013-7a04-4e16-895d-4e91d25e3a02 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:22.527716 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c447f013-7a04-4e16-895d-4e91d25e3a02 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:22.532306 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441dbc0c-d4bb-45dc-84d0-e2b94aa5ec0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:22.532905 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441dbc0c-d4bb-45dc-84d0-e2b94aa5ec0c admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:22.533191 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441dbc0c-d4bb-45dc-84d0-e2b94aa5ec0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:22.533463 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441dbc0c-d4bb-45dc-84d0-e2b94aa5ec0c admin admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:22.536944 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441dbc0c-d4bb-45dc-84d0-e2b94aa5ec0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:32:22.536944 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441dbc0c-d4bb-45dc-84d0-e2b94aa5ec0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:22.536944 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441dbc0c-d4bb-45dc-84d0-e2b94aa5ec0c admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:22.548100 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1409/2835] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:32:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 17:32:22.552630 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf35481-741e-4af6-b1b7-c9b728c58f0b None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:22.562143 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf35481-741e-4af6-b1b7-c9b728c58f0b None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:22.566848 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fac432e-1efc-44c9-8091-dbd775d086eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:22.567471 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac432e-1efc-44c9-8091-dbd775d086eb admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:22.567746 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac432e-1efc-44c9-8091-dbd775d086eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:22.568024 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac432e-1efc-44c9-8091-dbd775d086eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:22.568634 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fac432e-1efc-44c9-8091-dbd775d086eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:22.570519 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fac432e-1efc-44c9-8091-dbd775d086eb admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:22.671727 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1427/2836] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:32:22 2022] POST /identity/v3/projects => generated 377 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:32:22.676053 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441dbc0c-d4bb-45dc-84d0-e2b94aa5ec0c admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:22.686259 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441dbc0c-d4bb-45dc-84d0-e2b94aa5ec0c admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:22.741749 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf9d6a-03bf-4734-abd1-ede771d6bef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:22.742358 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf9d6a-03bf-4734-abd1-ede771d6bef6 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:22.742642 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf9d6a-03bf-4734-abd1-ede771d6bef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:22.742987 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf9d6a-03bf-4734-abd1-ede771d6bef6 admin admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:22.743892 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acf9d6a-03bf-4734-abd1-ede771d6bef6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:22.745806 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acf9d6a-03bf-4734-abd1-ede771d6bef6 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:22.753517 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:32:22.857357 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1410/2837] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:32:22 2022] POST /identity/v3/users => generated 289 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 17:32:22.861416 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fac432e-1efc-44c9-8091-dbd775d086eb admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:22.871995 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fac432e-1efc-44c9-8091-dbd775d086eb admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:22.876622 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526b663b-2273-472f-a09a-79fd528e59e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:22.877214 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526b663b-2273-472f-a09a-79fd528e59e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:22.877494 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526b663b-2273-472f-a09a-79fd528e59e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:22.877765 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526b663b-2273-472f-a09a-79fd528e59e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:22.878399 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526b663b-2273-472f-a09a-79fd528e59e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:22.880021 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526b663b-2273-472f-a09a-79fd528e59e5 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:22.890028 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1428/2838] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:32:22.893970 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf9d6a-03bf-4734-abd1-ede771d6bef6 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:22.904656 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf9d6a-03bf-4734-abd1-ede771d6bef6 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:22.909435 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac6988-73fe-443d-9aad-9b2c571d4e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:22.910088 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac6988-73fe-443d-9aad-9b2c571d4e1f admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:22.910398 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac6988-73fe-443d-9aad-9b2c571d4e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:22.910670 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac6988-73fe-443d-9aad-9b2c571d4e1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:22.911346 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceac6988-73fe-443d-9aad-9b2c571d4e1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:22.912957 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceac6988-73fe-443d-9aad-9b2c571d4e1f admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:22.924551 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1411/2839] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:32:22.928699 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526b663b-2273-472f-a09a-79fd528e59e5 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:22.938610 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526b663b-2273-472f-a09a-79fd528e59e5 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:22.943250 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eefa148-1b10-4789-b6a3-e9cc26aa9f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:22.943892 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eefa148-1b10-4789-b6a3-e9cc26aa9f16 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:22.944174 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eefa148-1b10-4789-b6a3-e9cc26aa9f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:22.944447 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eefa148-1b10-4789-b6a3-e9cc26aa9f16 admin admin] PATH_INFO: `/v3/projects/7a9855d2c2f34f8eb1917f8093d79dee/users/f7d4d2d1e4fb47609a8b2e1f79ddff28/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:22.958720 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eefa148-1b10-4789-b6a3-e9cc26aa9f16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a9855d2c2f34f8eb1917f8093d79dee, user_id=f7d4d2d1e4fb47609a8b2e1f79ddff28, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:22.961380 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eefa148-1b10-4789-b6a3-e9cc26aa9f16 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:23.056929 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1429/2840] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:32:22 2022] PUT /identity/v3/projects/7a9855d2c2f34f8eb1917f8093d79dee/users/f7d4d2d1e4fb47609a8b2e1f79ddff28/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:32:23.067284 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedb527b-97df-4d37-a0b0-0b79b9baf51e None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:23.067569 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedb527b-97df-4d37-a0b0-0b79b9baf51e None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:23.067872 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedb527b-97df-4d37-a0b0-0b79b9baf51e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:23.096444 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-bedb527b-97df-4d37-a0b0-0b79b9baf51e None None] MFA Rules not processed for user `f7d4d2d1e4fb47609a8b2e1f79ddff28`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:32:23.097638 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-bedb527b-97df-4d37-a0b0-0b79b9baf51e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:32:23.160196 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1412/2841] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:32:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:32:23.171826 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eefa148-1b10-4789-b6a3-e9cc26aa9f16 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:23.173552 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-5eefa148-1b10-4789-b6a3-e9cc26aa9f16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:32:23.251325 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eefa148-1b10-4789-b6a3-e9cc26aa9f16 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:23.254963 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc9b214-3519-427e-b265-a8f23b733158 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bee1ba6bb324de4915904f83349b3c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:23.255566 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc9b214-3519-427e-b265-a8f23b733158 service glance] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:23.255849 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc9b214-3519-427e-b265-a8f23b733158 service glance] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:23.256124 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc9b214-3519-427e-b265-a8f23b733158 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:23.264888 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc9b214-3519-427e-b265-a8f23b733158 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:23.266308 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc9b214-3519-427e-b265-a8f23b733158 service glance] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:23.276074 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1430/2842] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:32:23 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:32:33.157345 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:INSERT=3,keystone:SELECT=39 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:32:33.271037 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=82,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:32:46.717971 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedb527b-97df-4d37-a0b0-0b79b9baf51e None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:46.779335 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedb527b-97df-4d37-a0b0-0b79b9baf51e None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:46.784371 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c68838d-ce78-4f56-8ef9-7a413dd573bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:46.785102 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68838d-ce78-4f56-8ef9-7a413dd573bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:46.785339 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68838d-ce78-4f56-8ef9-7a413dd573bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:46.785615 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68838d-ce78-4f56-8ef9-7a413dd573bc admin admin] PATH_INFO: `/v3/users/f7d4d2d1e4fb47609a8b2e1f79ddff28` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:46.786738 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c68838d-ce78-4f56-8ef9-7a413dd573bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7d4d2d1e4fb47609a8b2e1f79ddff28)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:46.788684 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c68838d-ce78-4f56-8ef9-7a413dd573bc admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:46.968576 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-4c68838d-ce78-4f56-8ef9-7a413dd573bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7d4d2d1e4fb47609a8b2e1f79ddff28'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:46.980816 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-4c68838d-ce78-4f56-8ef9-7a413dd573bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7d4d2d1e4fb47609a8b2e1f79ddff28'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:46.981486 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-4c68838d-ce78-4f56-8ef9-7a413dd573bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7d4d2d1e4fb47609a8b2e1f79ddff28'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:47.051983 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-4c68838d-ce78-4f56-8ef9-7a413dd573bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7d4d2d1e4fb47609a8b2e1f79ddff28'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:47.052598 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-4c68838d-ce78-4f56-8ef9-7a413dd573bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7d4d2d1e4fb47609a8b2e1f79ddff28'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:47.064938 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1413/2843] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:32:46 2022] DELETE /identity/v3/users/f7d4d2d1e4fb47609a8b2e1f79ddff28 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:32:47.069418 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895a237-3833-4779-bdad-f2e05fbc7b41 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:47.075309 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895a237-3833-4779-bdad-f2e05fbc7b41 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:47.075309 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895a237-3833-4779-bdad-f2e05fbc7b41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:47.090233 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-f895a237-3833-4779-bdad-f2e05fbc7b41 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:32:47.091466 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-f895a237-3833-4779-bdad-f2e05fbc7b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:32:47.150676 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1431/2844] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:32:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:32:47.160370 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c68838d-ce78-4f56-8ef9-7a413dd573bc admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:47.161751 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-4c68838d-ce78-4f56-8ef9-7a413dd573bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:32:47.221454 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c68838d-ce78-4f56-8ef9-7a413dd573bc admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:47.225183 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cef32a7-386b-47af-8a3f-351865bd2819 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:47.225787 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef32a7-386b-47af-8a3f-351865bd2819 service neutron] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:47.226065 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef32a7-386b-47af-8a3f-351865bd2819 service neutron] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:47.226360 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef32a7-386b-47af-8a3f-351865bd2819 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:47.235115 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cef32a7-386b-47af-8a3f-351865bd2819 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:47.236551 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cef32a7-386b-47af-8a3f-351865bd2819 service neutron] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:47.246159 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1414/2845] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:32:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 17:32:47.653797 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f895a237-3833-4779-bdad-f2e05fbc7b41 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:47.655368 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-f895a237-3833-4779-bdad-f2e05fbc7b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:32:47.667640 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f895a237-3833-4779-bdad-f2e05fbc7b41 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:47.673207 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d39ec3-5ab4-47be-aa8a-80719cea5f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:47.673207 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d39ec3-5ab4-47be-aa8a-80719cea5f2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:47.673445 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d39ec3-5ab4-47be-aa8a-80719cea5f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:47.673601 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d39ec3-5ab4-47be-aa8a-80719cea5f2a admin admin] PATH_INFO: `/v3/projects/7a9855d2c2f34f8eb1917f8093d79dee` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:47.675505 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d39ec3-5ab4-47be-aa8a-80719cea5f2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a9855d2c2f34f8eb1917f8093d79dee)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:47.677411 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d39ec3-5ab4-47be-aa8a-80719cea5f2a admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:47.807724 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-c7d39ec3-5ab4-47be-aa8a-80719cea5f2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a9855d2c2f34f8eb1917f8093d79dee', 'request_id': 'req-c7d39ec3-5ab4-47be-aa8a-80719cea5f2a'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:47.814042 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-c7d39ec3-5ab4-47be-aa8a-80719cea5f2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a9855d2c2f34f8eb1917f8093d79dee', 'request_id': 'req-c7d39ec3-5ab4-47be-aa8a-80719cea5f2a'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:47.892165 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-c7d39ec3-5ab4-47be-aa8a-80719cea5f2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a9855d2c2f34f8eb1917f8093d79dee', 'request_id': 'req-c7d39ec3-5ab4-47be-aa8a-80719cea5f2a'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:47.902678 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-c7d39ec3-5ab4-47be-aa8a-80719cea5f2a admin admin] The token cache is being invalidate because project 7a9855d2c2f34f8eb1917f8093d79dee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:32:47.902990 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-c7d39ec3-5ab4-47be-aa8a-80719cea5f2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:47.903618 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-c7d39ec3-5ab4-47be-aa8a-80719cea5f2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:47.905163 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1432/2846] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:32:47 2022] DELETE /identity/v3/projects/7a9855d2c2f34f8eb1917f8093d79dee => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:32:47.910560 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f1687-5db7-4897-90ef-82e2dc740692 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:47.911066 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f1687-5db7-4897-90ef-82e2dc740692 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:47.911167 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f1687-5db7-4897-90ef-82e2dc740692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:47.930863 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-e38f1687-5db7-4897-90ef-82e2dc740692 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:32:47.932089 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-e38f1687-5db7-4897-90ef-82e2dc740692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:32:47.948509 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:32:47.991981 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1415/2847] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:32:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:32:47.999805 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4120a9bf-95f2-435c-aa5a-063f8de13c00 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:48.000096 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4120a9bf-95f2-435c-aa5a-063f8de13c00 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:48.000375 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4120a9bf-95f2-435c-aa5a-063f8de13c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:48.020093 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-4120a9bf-95f2-435c-aa5a-063f8de13c00 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:32:48.021240 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-4120a9bf-95f2-435c-aa5a-063f8de13c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:32:48.029324 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1433/2848] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:32:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:32:48.033439 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38f1687-5db7-4897-90ef-82e2dc740692 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:48.043598 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38f1687-5db7-4897-90ef-82e2dc740692 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:48.048624 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482e5539-a943-4b86-b24d-d4983d4b67aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:48.049220 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e5539-a943-4b86-b24d-d4983d4b67aa admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:48.050723 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e5539-a943-4b86-b24d-d4983d4b67aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:48.051030 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e5539-a943-4b86-b24d-d4983d4b67aa admin admin] PATH_INFO: `/v3/domains` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:48.051701 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482e5539-a943-4b86-b24d-d4983d4b67aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81650) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:32:48.052032 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482e5539-a943-4b86-b24d-d4983d4b67aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:48.053624 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482e5539-a943-4b86-b24d-d4983d4b67aa admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:48.065406 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1416/2849] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:32:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 17:32:48.069854 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4120a9bf-95f2-435c-aa5a-063f8de13c00 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:48.079375 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4120a9bf-95f2-435c-aa5a-063f8de13c00 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:48.084361 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c2f807-e1c5-45f1-a4a2-b5def164dd3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:48.084962 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c2f807-e1c5-45f1-a4a2-b5def164dd3e admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:48.085241 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c2f807-e1c5-45f1-a4a2-b5def164dd3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:48.085692 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c2f807-e1c5-45f1-a4a2-b5def164dd3e admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:48.086353 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c2f807-e1c5-45f1-a4a2-b5def164dd3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:48.088324 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c2f807-e1c5-45f1-a4a2-b5def164dd3e admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:48.191103 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1434/2850] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:32:48 2022] POST /identity/v3/projects => generated 373 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:32:48.195331 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482e5539-a943-4b86-b24d-d4983d4b67aa admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:48.205260 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482e5539-a943-4b86-b24d-d4983d4b67aa admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:48.260536 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae374340-fc83-40bb-b9b5-3481dff5b069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:48.261149 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae374340-fc83-40bb-b9b5-3481dff5b069 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:48.261430 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae374340-fc83-40bb-b9b5-3481dff5b069 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:48.261702 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae374340-fc83-40bb-b9b5-3481dff5b069 admin admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:48.262360 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae374340-fc83-40bb-b9b5-3481dff5b069 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:48.264312 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae374340-fc83-40bb-b9b5-3481dff5b069 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:48.370014 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1417/2851] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:32:48 2022] POST /identity/v3/users => generated 287 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:32:48.374302 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c2f807-e1c5-45f1-a4a2-b5def164dd3e admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:48.385002 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c2f807-e1c5-45f1-a4a2-b5def164dd3e admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:48.389607 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dcbe1c-7f92-42fb-a5a6-45259a508b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:48.390197 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dcbe1c-7f92-42fb-a5a6-45259a508b03 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:48.390473 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dcbe1c-7f92-42fb-a5a6-45259a508b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:48.390740 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dcbe1c-7f92-42fb-a5a6-45259a508b03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:48.391405 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dcbe1c-7f92-42fb-a5a6-45259a508b03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:48.393565 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dcbe1c-7f92-42fb-a5a6-45259a508b03 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:48.403402 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1435/2852] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:32:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:32:48.407258 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae374340-fc83-40bb-b9b5-3481dff5b069 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:48.417834 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae374340-fc83-40bb-b9b5-3481dff5b069 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:48.422409 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d37931c-ec2b-4677-8019-a5560a93caa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:48.423032 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d37931c-ec2b-4677-8019-a5560a93caa1 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:48.423314 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d37931c-ec2b-4677-8019-a5560a93caa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:48.423587 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d37931c-ec2b-4677-8019-a5560a93caa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:48.424216 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d37931c-ec2b-4677-8019-a5560a93caa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:48.426301 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d37931c-ec2b-4677-8019-a5560a93caa1 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:48.436519 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1418/2853] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:32:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:32:48.440810 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dcbe1c-7f92-42fb-a5a6-45259a508b03 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:48.450512 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dcbe1c-7f92-42fb-a5a6-45259a508b03 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:48.455065 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2987a8a1-e89e-413d-9038-6f42d3f139f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:48.456227 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2987a8a1-e89e-413d-9038-6f42d3f139f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:48.456733 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2987a8a1-e89e-413d-9038-6f42d3f139f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:48.457056 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2987a8a1-e89e-413d-9038-6f42d3f139f9 admin admin] PATH_INFO: `/v3/projects/afd2d3fb4f4543f6a4f04c1d4333d99a/users/8d1894666f29481e967307d2fa5af642/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:48.471106 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2987a8a1-e89e-413d-9038-6f42d3f139f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afd2d3fb4f4543f6a4f04c1d4333d99a, user_id=8d1894666f29481e967307d2fa5af642, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:48.474784 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2987a8a1-e89e-413d-9038-6f42d3f139f9 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:48.571292 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1436/2854] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:32:48 2022] PUT /identity/v3/projects/afd2d3fb4f4543f6a4f04c1d4333d99a/users/8d1894666f29481e967307d2fa5af642/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:32:48.582269 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256b9900-0b74-4413-b53e-401e0fcd831e None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:48.582559 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256b9900-0b74-4413-b53e-401e0fcd831e None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:48.582836 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256b9900-0b74-4413-b53e-401e0fcd831e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:48.610585 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-256b9900-0b74-4413-b53e-401e0fcd831e None None] MFA Rules not processed for user `8d1894666f29481e967307d2fa5af642`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:32:48.611745 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-256b9900-0b74-4413-b53e-401e0fcd831e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:32:48.674189 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1419/2855] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:32:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 17:32:48.679324 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2987a8a1-e89e-413d-9038-6f42d3f139f9 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:48.700663 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:32:48.740747 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2987a8a1-e89e-413d-9038-6f42d3f139f9 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:48.745369 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf18ae6f-6b44-4180-ae96-ca57f51c8a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:48.746015 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf18ae6f-6b44-4180-ae96-ca57f51c8a0d admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:48.746295 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf18ae6f-6b44-4180-ae96-ca57f51c8a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:48.746567 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf18ae6f-6b44-4180-ae96-ca57f51c8a0d admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:48.747229 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf18ae6f-6b44-4180-ae96-ca57f51c8a0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:48.749147 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf18ae6f-6b44-4180-ae96-ca57f51c8a0d admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:48.874296 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1437/2856] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:32:48 2022] POST /identity/v3/projects => generated 375 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:32:48.878872 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256b9900-0b74-4413-b53e-401e0fcd831e None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:48.889467 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256b9900-0b74-4413-b53e-401e0fcd831e None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:48.902012 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:32:48.947366 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ecdd5-d1bc-4759-8852-0564c11fcded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:48.948005 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ecdd5-d1bc-4759-8852-0564c11fcded admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:48.948233 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ecdd5-d1bc-4759-8852-0564c11fcded admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:48.948505 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ecdd5-d1bc-4759-8852-0564c11fcded admin admin] PATH_INFO: `/v3/users` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:48.949118 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5ecdd5-d1bc-4759-8852-0564c11fcded admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:48.951070 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5ecdd5-d1bc-4759-8852-0564c11fcded admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:49.057779 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1420/2857] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:32:48 2022] POST /identity/v3/users => generated 294 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:32:49.061922 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf18ae6f-6b44-4180-ae96-ca57f51c8a0d admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:49.072887 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf18ae6f-6b44-4180-ae96-ca57f51c8a0d admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:49.077518 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca99e759-de6f-4fbb-a561-a9164a42f5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:49.078115 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca99e759-de6f-4fbb-a561-a9164a42f5a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:49.078398 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca99e759-de6f-4fbb-a561-a9164a42f5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:49.078672 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca99e759-de6f-4fbb-a561-a9164a42f5a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:49.079335 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca99e759-de6f-4fbb-a561-a9164a42f5a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:49.081201 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca99e759-de6f-4fbb-a561-a9164a42f5a8 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:49.091703 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1438/2858] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:32:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:32:49.095793 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ecdd5-d1bc-4759-8852-0564c11fcded admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:49.105606 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ecdd5-d1bc-4759-8852-0564c11fcded admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:49.110081 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926a87ec-1ae9-4212-8970-e43d49f4d50a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:49.110723 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926a87ec-1ae9-4212-8970-e43d49f4d50a admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:49.111037 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926a87ec-1ae9-4212-8970-e43d49f4d50a admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:49.111313 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926a87ec-1ae9-4212-8970-e43d49f4d50a admin admin] PATH_INFO: `/v3/projects/f1de290ad7324a69b132bf27e1bbf0a5/users/64e1307556da4607bf86dd2a95b6ef16/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:49.126191 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926a87ec-1ae9-4212-8970-e43d49f4d50a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1de290ad7324a69b132bf27e1bbf0a5, user_id=64e1307556da4607bf86dd2a95b6ef16, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:49.129593 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926a87ec-1ae9-4212-8970-e43d49f4d50a admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:49.205011 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1421/2859] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:32:49 2022] PUT /identity/v3/projects/f1de290ad7324a69b132bf27e1bbf0a5/users/64e1307556da4607bf86dd2a95b6ef16/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:32:49.209189 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca99e759-de6f-4fbb-a561-a9164a42f5a8 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:49.219051 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca99e759-de6f-4fbb-a561-a9164a42f5a8 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:49.276147 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a828183a-ecb9-42db-ba32-a07d5976454c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:49.276823 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828183a-ecb9-42db-ba32-a07d5976454c admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:49.277075 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828183a-ecb9-42db-ba32-a07d5976454c admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:49.277296 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828183a-ecb9-42db-ba32-a07d5976454c admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:49.277928 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a828183a-ecb9-42db-ba32-a07d5976454c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:49.279567 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a828183a-ecb9-42db-ba32-a07d5976454c admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:49.289910 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1439/2860] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:32:49 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:32:49.294200 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926a87ec-1ae9-4212-8970-e43d49f4d50a admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:49.305637 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926a87ec-1ae9-4212-8970-e43d49f4d50a admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:49.310127 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeadc3e-80ae-4dfb-be58-3214898ada47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:49.310774 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeadc3e-80ae-4dfb-be58-3214898ada47 admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:49.311089 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeadc3e-80ae-4dfb-be58-3214898ada47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:49.311360 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeadc3e-80ae-4dfb-be58-3214898ada47 admin admin] PATH_INFO: `/v3/projects/f1de290ad7324a69b132bf27e1bbf0a5/users/64e1307556da4607bf86dd2a95b6ef16/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:49.313278 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeadc3e-80ae-4dfb-be58-3214898ada47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1de290ad7324a69b132bf27e1bbf0a5, user_id=64e1307556da4607bf86dd2a95b6ef16, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:49.316959 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeadc3e-80ae-4dfb-be58-3214898ada47 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:49.327568 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1422/2861] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:32:49 2022] PUT /identity/v3/projects/f1de290ad7324a69b132bf27e1bbf0a5/users/64e1307556da4607bf86dd2a95b6ef16/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:32:49.339365 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ebe46-cd42-4ce0-b8e0-9310d1bb6694 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:49.339659 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ebe46-cd42-4ce0-b8e0-9310d1bb6694 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:49.339961 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ebe46-cd42-4ce0-b8e0-9310d1bb6694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:49.368181 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-8a7ebe46-cd42-4ce0-b8e0-9310d1bb6694 None None] MFA Rules not processed for user `64e1307556da4607bf86dd2a95b6ef16`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:32:49.369398 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-8a7ebe46-cd42-4ce0-b8e0-9310d1bb6694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:32:49.441675 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1440/2862] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:32:49 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:32:49.452537 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeadc3e-80ae-4dfb-be58-3214898ada47 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:49.454274 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-5eeadc3e-80ae-4dfb-be58-3214898ada47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:32:49.513868 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeadc3e-80ae-4dfb-be58-3214898ada47 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:49.517536 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e1900a-f04d-4914-9c3f-b794b42c38b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:49.518310 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e1900a-f04d-4914-9c3f-b794b42c38b3 service cinder] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:49.518593 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e1900a-f04d-4914-9c3f-b794b42c38b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:49.518868 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e1900a-f04d-4914-9c3f-b794b42c38b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:49.575565 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e1900a-f04d-4914-9c3f-b794b42c38b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:49.577025 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e1900a-f04d-4914-9c3f-b794b42c38b3 service cinder] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:49.586515 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1423/2863] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:32:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:32:52.706242 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7ebe46-cd42-4ce0-b8e0-9310d1bb6694 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:52.723991 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:32:52.767594 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7ebe46-cd42-4ce0-b8e0-9310d1bb6694 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:52.772889 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16503fae-fbe9-4233-a5f2-adf7272b7db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:52.773523 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16503fae-fbe9-4233-a5f2-adf7272b7db1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:52.773802 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16503fae-fbe9-4233-a5f2-adf7272b7db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:52.774081 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16503fae-fbe9-4233-a5f2-adf7272b7db1 admin admin] PATH_INFO: `/v3/users/8d1894666f29481e967307d2fa5af642` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:52.775220 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16503fae-fbe9-4233-a5f2-adf7272b7db1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d1894666f29481e967307d2fa5af642)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:52.777172 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16503fae-fbe9-4233-a5f2-adf7272b7db1 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:52.931927 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-16503fae-fbe9-4233-a5f2-adf7272b7db1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d1894666f29481e967307d2fa5af642'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:52.989404 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-16503fae-fbe9-4233-a5f2-adf7272b7db1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d1894666f29481e967307d2fa5af642'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:52.989981 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-16503fae-fbe9-4233-a5f2-adf7272b7db1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d1894666f29481e967307d2fa5af642'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:52.990600 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-16503fae-fbe9-4233-a5f2-adf7272b7db1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d1894666f29481e967307d2fa5af642'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:52.997869 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-16503fae-fbe9-4233-a5f2-adf7272b7db1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d1894666f29481e967307d2fa5af642'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:53.014880 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1441/2864] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:32:52 2022] DELETE /identity/v3/users/8d1894666f29481e967307d2fa5af642 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 17:32:53.020175 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41812b1-acb7-4e31-ab9d-0849041fdf69 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:53.020658 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41812b1-acb7-4e31-ab9d-0849041fdf69 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:53.020761 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41812b1-acb7-4e31-ab9d-0849041fdf69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:53.040446 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-c41812b1-acb7-4e31-ab9d-0849041fdf69 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:32:53.041633 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-c41812b1-acb7-4e31-ab9d-0849041fdf69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:32:53.059570 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:32:53.100737 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1424/2865] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:32:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:32:53.110715 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16503fae-fbe9-4233-a5f2-adf7272b7db1 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:53.112464 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-16503fae-fbe9-4233-a5f2-adf7272b7db1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:32:53.169998 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16503fae-fbe9-4233-a5f2-adf7272b7db1 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:53.173637 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963b2ca9-0b42-43d5-9774-cb30bf4dc545 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:53.174239 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963b2ca9-0b42-43d5-9774-cb30bf4dc545 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:53.174520 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963b2ca9-0b42-43d5-9774-cb30bf4dc545 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:53.174795 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963b2ca9-0b42-43d5-9774-cb30bf4dc545 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:53.183787 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963b2ca9-0b42-43d5-9774-cb30bf4dc545 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:53.185200 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963b2ca9-0b42-43d5-9774-cb30bf4dc545 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:53.194967 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1442/2866] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:32:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:32:53.597279 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41812b1-acb7-4e31-ab9d-0849041fdf69 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:53.598736 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-c41812b1-acb7-4e31-ab9d-0849041fdf69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:32:53.610420 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41812b1-acb7-4e31-ab9d-0849041fdf69 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:53.615982 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8ef031-e2ab-4992-b500-135db7ffcc5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:53.616614 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ef031-e2ab-4992-b500-135db7ffcc5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:53.616894 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ef031-e2ab-4992-b500-135db7ffcc5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:53.617221 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ef031-e2ab-4992-b500-135db7ffcc5c admin admin] PATH_INFO: `/v3/projects/afd2d3fb4f4543f6a4f04c1d4333d99a` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:53.618326 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8ef031-e2ab-4992-b500-135db7ffcc5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=afd2d3fb4f4543f6a4f04c1d4333d99a)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:53.620277 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8ef031-e2ab-4992-b500-135db7ffcc5c admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:53.715426 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-5c8ef031-e2ab-4992-b500-135db7ffcc5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afd2d3fb4f4543f6a4f04c1d4333d99a', 'request_id': 'req-5c8ef031-e2ab-4992-b500-135db7ffcc5c'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:53.772631 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-5c8ef031-e2ab-4992-b500-135db7ffcc5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afd2d3fb4f4543f6a4f04c1d4333d99a', 'request_id': 'req-5c8ef031-e2ab-4992-b500-135db7ffcc5c'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:53.778440 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-5c8ef031-e2ab-4992-b500-135db7ffcc5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afd2d3fb4f4543f6a4f04c1d4333d99a', 'request_id': 'req-5c8ef031-e2ab-4992-b500-135db7ffcc5c'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:53.790192 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-5c8ef031-e2ab-4992-b500-135db7ffcc5c admin admin] The token cache is being invalidate because project afd2d3fb4f4543f6a4f04c1d4333d99a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81650) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:32:53.790478 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-5c8ef031-e2ab-4992-b500-135db7ffcc5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:53.791150 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-5c8ef031-e2ab-4992-b500-135db7ffcc5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:53.792675 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1425/2867] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:32:53 2022] DELETE /identity/v3/projects/afd2d3fb4f4543f6a4f04c1d4333d99a => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:32:53.796667 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963b2ca9-0b42-43d5-9774-cb30bf4dc545 service neutron] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:53.807417 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963b2ca9-0b42-43d5-9774-cb30bf4dc545 service neutron] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:53.840768 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:32:53.863313 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5067a1b6-5d45-4303-9ef6-6a2ad905dd6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:53.863932 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5067a1b6-5d45-4303-9ef6-6a2ad905dd6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:53.864216 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5067a1b6-5d45-4303-9ef6-6a2ad905dd6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:53.864491 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5067a1b6-5d45-4303-9ef6-6a2ad905dd6b admin admin] PATH_INFO: `/v3/users/64e1307556da4607bf86dd2a95b6ef16` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:53.865601 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5067a1b6-5d45-4303-9ef6-6a2ad905dd6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=64e1307556da4607bf86dd2a95b6ef16)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:53.867549 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5067a1b6-5d45-4303-9ef6-6a2ad905dd6b admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:54.048736 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-5067a1b6-5d45-4303-9ef6-6a2ad905dd6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64e1307556da4607bf86dd2a95b6ef16'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:54.115138 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-5067a1b6-5d45-4303-9ef6-6a2ad905dd6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64e1307556da4607bf86dd2a95b6ef16'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:54.115704 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-5067a1b6-5d45-4303-9ef6-6a2ad905dd6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64e1307556da4607bf86dd2a95b6ef16'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:54.116303 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-5067a1b6-5d45-4303-9ef6-6a2ad905dd6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64e1307556da4607bf86dd2a95b6ef16'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:54.123326 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-5067a1b6-5d45-4303-9ef6-6a2ad905dd6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64e1307556da4607bf86dd2a95b6ef16'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:54.140820 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1443/2868] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:32:53 2022] DELETE /identity/v3/users/64e1307556da4607bf86dd2a95b6ef16 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:32:54.534505 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8ef031-e2ab-4992-b500-135db7ffcc5c admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:54.538497 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-5c8ef031-e2ab-4992-b500-135db7ffcc5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:32:54.550538 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8ef031-e2ab-4992-b500-135db7ffcc5c admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:54.608005 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98d4d67-f9ff-4b72-aeec-dbbc2220ba8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:54.608641 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d4d67-f9ff-4b72-aeec-dbbc2220ba8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:54.608925 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d4d67-f9ff-4b72-aeec-dbbc2220ba8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:54.609198 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d4d67-f9ff-4b72-aeec-dbbc2220ba8b admin admin] PATH_INFO: `/v3/projects/f1de290ad7324a69b132bf27e1bbf0a5` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:54.610301 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98d4d67-f9ff-4b72-aeec-dbbc2220ba8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1de290ad7324a69b132bf27e1bbf0a5)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:54.612534 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98d4d67-f9ff-4b72-aeec-dbbc2220ba8b admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:54.704500 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-f98d4d67-f9ff-4b72-aeec-dbbc2220ba8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1de290ad7324a69b132bf27e1bbf0a5', 'request_id': 'req-f98d4d67-f9ff-4b72-aeec-dbbc2220ba8b'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:54.706758 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:32:54.830372 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-f98d4d67-f9ff-4b72-aeec-dbbc2220ba8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1de290ad7324a69b132bf27e1bbf0a5', 'request_id': 'req-f98d4d67-f9ff-4b72-aeec-dbbc2220ba8b'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:54.837118 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-f98d4d67-f9ff-4b72-aeec-dbbc2220ba8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1de290ad7324a69b132bf27e1bbf0a5', 'request_id': 'req-f98d4d67-f9ff-4b72-aeec-dbbc2220ba8b'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:54.848752 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-f98d4d67-f9ff-4b72-aeec-dbbc2220ba8b admin admin] The token cache is being invalidate because project f1de290ad7324a69b132bf27e1bbf0a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81650) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:32:54.849039 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-f98d4d67-f9ff-4b72-aeec-dbbc2220ba8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:54.849674 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-f98d4d67-f9ff-4b72-aeec-dbbc2220ba8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:32:54.851295 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1426/2869] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:32:54 2022] DELETE /identity/v3/projects/f1de290ad7324a69b132bf27e1bbf0a5 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:32:54.856154 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab668d4-0cc1-40d0-9712-8f9108ad2e37 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:54.856645 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab668d4-0cc1-40d0-9712-8f9108ad2e37 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:54.856756 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab668d4-0cc1-40d0-9712-8f9108ad2e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:54.876544 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-aab668d4-0cc1-40d0-9712-8f9108ad2e37 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:32:54.878293 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-aab668d4-0cc1-40d0-9712-8f9108ad2e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:32:54.937306 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1444/2870] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:32:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:32:54.945854 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a075b-4df1-4e33-b5ae-cf21b654aeb7 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:54.946140 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a075b-4df1-4e33-b5ae-cf21b654aeb7 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:54.946415 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a075b-4df1-4e33-b5ae-cf21b654aeb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:54.966252 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-701a075b-4df1-4e33-b5ae-cf21b654aeb7 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:32:54.967448 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-701a075b-4df1-4e33-b5ae-cf21b654aeb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:32:54.975338 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1427/2871] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:32:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:32:54.979702 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab668d4-0cc1-40d0-9712-8f9108ad2e37 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:54.990075 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab668d4-0cc1-40d0-9712-8f9108ad2e37 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:54.994608 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4081de33-bf5a-4ced-87fc-30e8ca51af8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:54.995236 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4081de33-bf5a-4ced-87fc-30e8ca51af8e admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:54.995517 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4081de33-bf5a-4ced-87fc-30e8ca51af8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:55.000502 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4081de33-bf5a-4ced-87fc-30e8ca51af8e admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:55.000502 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4081de33-bf5a-4ced-87fc-30e8ca51af8e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:32:55.000502 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4081de33-bf5a-4ced-87fc-30e8ca51af8e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:55.000502 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4081de33-bf5a-4ced-87fc-30e8ca51af8e admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:55.012066 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1445/2872] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:32:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 17:32:55.016330 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701a075b-4df1-4e33-b5ae-cf21b654aeb7 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:55.025581 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701a075b-4df1-4e33-b5ae-cf21b654aeb7 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:55.030279 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3691fe-ed91-4f84-84f1-cf6cabd3d305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:55.030880 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3691fe-ed91-4f84-84f1-cf6cabd3d305 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:55.031192 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3691fe-ed91-4f84-84f1-cf6cabd3d305 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:55.031468 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3691fe-ed91-4f84-84f1-cf6cabd3d305 admin admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:55.032088 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3691fe-ed91-4f84-84f1-cf6cabd3d305 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:55.034039 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3691fe-ed91-4f84-84f1-cf6cabd3d305 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:55.093807 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1428/2873] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:32:55 2022] POST /identity/v3/projects => generated 375 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:32:55.098382 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4081de33-bf5a-4ced-87fc-30e8ca51af8e admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:55.108314 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4081de33-bf5a-4ced-87fc-30e8ca51af8e admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:55.150014 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:32:55.164328 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1123eda8-9e14-4f5b-96ed-c32eea36a863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:55.164937 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1123eda8-9e14-4f5b-96ed-c32eea36a863 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:55.165219 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1123eda8-9e14-4f5b-96ed-c32eea36a863 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:55.165493 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1123eda8-9e14-4f5b-96ed-c32eea36a863 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:55.166179 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1123eda8-9e14-4f5b-96ed-c32eea36a863 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:55.168125 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1123eda8-9e14-4f5b-96ed-c32eea36a863 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:55.252480 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1446/2874] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:32:55 2022] POST /identity/v3/users => generated 288 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:32:55.257215 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3691fe-ed91-4f84-84f1-cf6cabd3d305 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:55.267515 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3691fe-ed91-4f84-84f1-cf6cabd3d305 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:55.272202 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7caf68b-16a5-4f9c-88b1-237ec5379667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:55.272796 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7caf68b-16a5-4f9c-88b1-237ec5379667 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:55.273078 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7caf68b-16a5-4f9c-88b1-237ec5379667 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:55.273375 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7caf68b-16a5-4f9c-88b1-237ec5379667 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:55.274011 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7caf68b-16a5-4f9c-88b1-237ec5379667 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:55.275649 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7caf68b-16a5-4f9c-88b1-237ec5379667 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:55.285901 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1429/2875] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:32:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:32:55.289747 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1123eda8-9e14-4f5b-96ed-c32eea36a863 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:55.300146 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1123eda8-9e14-4f5b-96ed-c32eea36a863 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:55.305297 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2303c070-96e0-4684-aa42-ea19d363918f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:55.305892 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2303c070-96e0-4684-aa42-ea19d363918f admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:55.306176 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2303c070-96e0-4684-aa42-ea19d363918f admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:55.306456 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2303c070-96e0-4684-aa42-ea19d363918f admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:55.307119 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2303c070-96e0-4684-aa42-ea19d363918f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:55.308723 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2303c070-96e0-4684-aa42-ea19d363918f admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:55.319953 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1447/2876] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:32:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:32:55.323974 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7caf68b-16a5-4f9c-88b1-237ec5379667 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:55.334332 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7caf68b-16a5-4f9c-88b1-237ec5379667 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:55.338800 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f6f012-4840-477d-9d71-bd02b51ec3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:55.339476 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6f012-4840-477d-9d71-bd02b51ec3ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:55.339752 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6f012-4840-477d-9d71-bd02b51ec3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:55.340022 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6f012-4840-477d-9d71-bd02b51ec3ac admin admin] PATH_INFO: `/v3/projects/d52b3725ffad4689b2aff79f2eea51d5/users/2ed5f8c2053548babdcdabf178353915/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:55.354213 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f6f012-4840-477d-9d71-bd02b51ec3ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=d52b3725ffad4689b2aff79f2eea51d5, user_id=2ed5f8c2053548babdcdabf178353915, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:55.356884 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f6f012-4840-477d-9d71-bd02b51ec3ac admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:55.433167 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1430/2877] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:32:55 2022] PUT /identity/v3/projects/d52b3725ffad4689b2aff79f2eea51d5/users/2ed5f8c2053548babdcdabf178353915/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:32:55.443953 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42218f79-ed74-47dd-8696-885cc418039f None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:55.444242 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42218f79-ed74-47dd-8696-885cc418039f None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:55.444518 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42218f79-ed74-47dd-8696-885cc418039f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:55.471955 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-42218f79-ed74-47dd-8696-885cc418039f None None] MFA Rules not processed for user `2ed5f8c2053548babdcdabf178353915`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:32:55.473183 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-42218f79-ed74-47dd-8696-885cc418039f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:32:55.537058 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1448/2878] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:32:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:32:55.547412 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f6f012-4840-477d-9d71-bd02b51ec3ac admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:32:55.549196 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-d8f6f012-4840-477d-9d71-bd02b51ec3ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:32:55.604517 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f6f012-4840-477d-9d71-bd02b51ec3ac admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:32:55.608096 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32846a1c-b8c2-4411-9c8a-142035c4eb71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85876a26a22e4c2c882883f7e978e4f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:32:55.608694 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32846a1c-b8c2-4411-9c8a-142035c4eb71 service cinder] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:32:55.608977 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32846a1c-b8c2-4411-9c8a-142035c4eb71 service cinder] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:32:55.609277 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32846a1c-b8c2-4411-9c8a-142035c4eb71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:32:55.617739 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32846a1c-b8c2-4411-9c8a-142035c4eb71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:32:55.619533 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32846a1c-b8c2-4411-9c8a-142035c4eb71 service cinder] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:32:55.628212 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1431/2879] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:32:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:33:05.533533 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=46,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:33:05.624195 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=59 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:33:07.185467 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42218f79-ed74-47dd-8696-885cc418039f None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:33:07.247118 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42218f79-ed74-47dd-8696-885cc418039f None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:33:07.251639 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9da6d7a-7af1-4a78-b681-8d1c7fe9b1a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:33:07.252276 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9da6d7a-7af1-4a78-b681-8d1c7fe9b1a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:33:07.252665 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9da6d7a-7af1-4a78-b681-8d1c7fe9b1a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:33:07.252935 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9da6d7a-7af1-4a78-b681-8d1c7fe9b1a0 admin admin] PATH_INFO: `/v3/users/2ed5f8c2053548babdcdabf178353915` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:33:07.254014 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9da6d7a-7af1-4a78-b681-8d1c7fe9b1a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ed5f8c2053548babdcdabf178353915)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:33:07.255962 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9da6d7a-7af1-4a78-b681-8d1c7fe9b1a0 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:33:07.423303 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-d9da6d7a-7af1-4a78-b681-8d1c7fe9b1a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ed5f8c2053548babdcdabf178353915'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:33:07.507363 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-d9da6d7a-7af1-4a78-b681-8d1c7fe9b1a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ed5f8c2053548babdcdabf178353915'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:33:07.508028 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-d9da6d7a-7af1-4a78-b681-8d1c7fe9b1a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ed5f8c2053548babdcdabf178353915'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:33:07.508648 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-d9da6d7a-7af1-4a78-b681-8d1c7fe9b1a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ed5f8c2053548babdcdabf178353915'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:33:07.515877 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-d9da6d7a-7af1-4a78-b681-8d1c7fe9b1a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ed5f8c2053548babdcdabf178353915'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:33:07.533401 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1449/2880] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:33:07 2022] DELETE /identity/v3/users/2ed5f8c2053548babdcdabf178353915 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:33:07.538079 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2263271f-4bd7-42b0-a5ac-ad049cd0f157 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:33:07.538569 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2263271f-4bd7-42b0-a5ac-ad049cd0f157 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:33:07.538680 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2263271f-4bd7-42b0-a5ac-ad049cd0f157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:33:07.557318 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-2263271f-4bd7-42b0-a5ac-ad049cd0f157 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:33:07.558520 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-2263271f-4bd7-42b0-a5ac-ad049cd0f157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:33:07.617503 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1432/2881] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:33:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:33:07.627341 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9da6d7a-7af1-4a78-b681-8d1c7fe9b1a0 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:33:07.629300 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-d9da6d7a-7af1-4a78-b681-8d1c7fe9b1a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:33:07.687785 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9da6d7a-7af1-4a78-b681-8d1c7fe9b1a0 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:33:07.691927 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380080da-ba3b-4d5a-9efc-f3d76ff7829f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:33:07.692525 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380080da-ba3b-4d5a-9efc-f3d76ff7829f service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:33:07.692801 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380080da-ba3b-4d5a-9efc-f3d76ff7829f service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:33:07.693069 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380080da-ba3b-4d5a-9efc-f3d76ff7829f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:33:07.701721 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380080da-ba3b-4d5a-9efc-f3d76ff7829f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:33:07.703163 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380080da-ba3b-4d5a-9efc-f3d76ff7829f service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:33:07.712988 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1450/2882] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:33:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:33:08.137742 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2263271f-4bd7-42b0-a5ac-ad049cd0f157 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:33:08.139328 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-2263271f-4bd7-42b0-a5ac-ad049cd0f157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:33:08.152345 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2263271f-4bd7-42b0-a5ac-ad049cd0f157 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:33:08.157045 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e80314-0813-45ef-b123-c74393c81409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:33:08.157547 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e80314-0813-45ef-b123-c74393c81409 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:33:08.157857 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e80314-0813-45ef-b123-c74393c81409 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:33:08.158137 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e80314-0813-45ef-b123-c74393c81409 admin admin] PATH_INFO: `/v3/projects/d52b3725ffad4689b2aff79f2eea51d5` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:33:08.162659 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e80314-0813-45ef-b123-c74393c81409 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d52b3725ffad4689b2aff79f2eea51d5)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:33:08.162659 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e80314-0813-45ef-b123-c74393c81409 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:33:08.296013 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-79e80314-0813-45ef-b123-c74393c81409 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd52b3725ffad4689b2aff79f2eea51d5', 'request_id': 'req-79e80314-0813-45ef-b123-c74393c81409'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:33:08.376197 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-79e80314-0813-45ef-b123-c74393c81409 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd52b3725ffad4689b2aff79f2eea51d5', 'request_id': 'req-79e80314-0813-45ef-b123-c74393c81409'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:33:08.382721 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-79e80314-0813-45ef-b123-c74393c81409 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd52b3725ffad4689b2aff79f2eea51d5', 'request_id': 'req-79e80314-0813-45ef-b123-c74393c81409'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:33:08.394274 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-79e80314-0813-45ef-b123-c74393c81409 admin admin] The token cache is being invalidate because project d52b3725ffad4689b2aff79f2eea51d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81650) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:33:08.394559 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-79e80314-0813-45ef-b123-c74393c81409 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:33:08.396827 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-79e80314-0813-45ef-b123-c74393c81409 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:33:08.396827 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1433/2883] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:33:08 2022] DELETE /identity/v3/projects/d52b3725ffad4689b2aff79f2eea51d5 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:33:08.401734 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b26a4-5cb3-481b-8e03-a5c6ccf1b7f7 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:33:08.402211 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b26a4-5cb3-481b-8e03-a5c6ccf1b7f7 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:33:08.402315 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b26a4-5cb3-481b-8e03-a5c6ccf1b7f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:33:08.421851 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-966b26a4-5cb3-481b-8e03-a5c6ccf1b7f7 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:33:08.425835 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-966b26a4-5cb3-481b-8e03-a5c6ccf1b7f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:33:08.440508 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:33:08.481859 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1451/2884] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:33:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:33:08.490124 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d9ff0-c91c-47d8-b04f-c66c0bfb0978 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:33:08.490461 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d9ff0-c91c-47d8-b04f-c66c0bfb0978 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:33:08.490795 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d9ff0-c91c-47d8-b04f-c66c0bfb0978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:33:08.511748 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-448d9ff0-c91c-47d8-b04f-c66c0bfb0978 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:33:08.512936 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-448d9ff0-c91c-47d8-b04f-c66c0bfb0978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:33:08.520838 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1434/2885] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:33:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:33:08.525224 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966b26a4-5cb3-481b-8e03-a5c6ccf1b7f7 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:33:08.534645 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966b26a4-5cb3-481b-8e03-a5c6ccf1b7f7 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:33:08.539146 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a4fd1-73eb-469a-aad9-e86f82c4e474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:33:08.539740 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a4fd1-73eb-469a-aad9-e86f82c4e474 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:33:08.540018 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a4fd1-73eb-469a-aad9-e86f82c4e474 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:33:08.540288 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a4fd1-73eb-469a-aad9-e86f82c4e474 admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:33:08.540921 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6a4fd1-73eb-469a-aad9-e86f82c4e474 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 17:33:08.541223 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6a4fd1-73eb-469a-aad9-e86f82c4e474 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:33:08.542809 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6a4fd1-73eb-469a-aad9-e86f82c4e474 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:33:08.555354 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1452/2886] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 17:33:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 17:33:08.559445 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448d9ff0-c91c-47d8-b04f-c66c0bfb0978 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:33:08.569187 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448d9ff0-c91c-47d8-b04f-c66c0bfb0978 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:33:08.574020 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df42cbbc-f5ee-4f8a-b628-92c2635106a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:33:08.574616 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42cbbc-f5ee-4f8a-b628-92c2635106a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:33:08.574922 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42cbbc-f5ee-4f8a-b628-92c2635106a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:33:08.575200 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42cbbc-f5ee-4f8a-b628-92c2635106a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:33:08.575843 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df42cbbc-f5ee-4f8a-b628-92c2635106a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:33:08.577755 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df42cbbc-f5ee-4f8a-b628-92c2635106a6 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:33:08.692292 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1435/2887] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:33:08 2022] POST /identity/v3/projects => generated 383 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:33:08.696692 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a4fd1-73eb-469a-aad9-e86f82c4e474 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:33:08.706770 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a4fd1-73eb-469a-aad9-e86f82c4e474 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:33:08.762119 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429cd849-5df3-4cd9-8b4e-25431bd5eb70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:33:08.762725 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429cd849-5df3-4cd9-8b4e-25431bd5eb70 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:33:08.763041 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429cd849-5df3-4cd9-8b4e-25431bd5eb70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:33:08.763323 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429cd849-5df3-4cd9-8b4e-25431bd5eb70 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:33:08.763959 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429cd849-5df3-4cd9-8b4e-25431bd5eb70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:33:08.765880 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429cd849-5df3-4cd9-8b4e-25431bd5eb70 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:33:08.880380 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1453/2888] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:33:08 2022] POST /identity/v3/users => generated 292 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:33:08.884532 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df42cbbc-f5ee-4f8a-b628-92c2635106a6 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:33:08.895217 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df42cbbc-f5ee-4f8a-b628-92c2635106a6 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:33:08.899688 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9a69c9-e5ba-4321-a60d-d10b4a44dab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:33:08.900947 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a69c9-e5ba-4321-a60d-d10b4a44dab8 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:33:08.900947 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a69c9-e5ba-4321-a60d-d10b4a44dab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:33:08.900947 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a69c9-e5ba-4321-a60d-d10b4a44dab8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:33:08.901431 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9a69c9-e5ba-4321-a60d-d10b4a44dab8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:33:08.903055 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9a69c9-e5ba-4321-a60d-d10b4a44dab8 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:33:08.914220 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1436/2889] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:33:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:33:08.918208 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429cd849-5df3-4cd9-8b4e-25431bd5eb70 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:33:08.929684 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429cd849-5df3-4cd9-8b4e-25431bd5eb70 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:33:08.935027 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff56a148-6433-4951-939c-045b385eed8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:33:08.935199 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff56a148-6433-4951-939c-045b385eed8e admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:33:08.935475 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff56a148-6433-4951-939c-045b385eed8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:33:08.935743 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff56a148-6433-4951-939c-045b385eed8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:33:08.936398 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff56a148-6433-4951-939c-045b385eed8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:33:08.938268 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff56a148-6433-4951-939c-045b385eed8e admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:33:08.949801 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1454/2890] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:33:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:33:08.953696 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9a69c9-e5ba-4321-a60d-d10b4a44dab8 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:33:08.964080 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9a69c9-e5ba-4321-a60d-d10b4a44dab8 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:33:08.968624 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cae083f-1a0c-4a36-8ccd-d6ecd4bca7bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:33:08.969256 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae083f-1a0c-4a36-8ccd-d6ecd4bca7bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:33:08.969531 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae083f-1a0c-4a36-8ccd-d6ecd4bca7bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:33:08.969799 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae083f-1a0c-4a36-8ccd-d6ecd4bca7bf admin admin] PATH_INFO: `/v3/projects/310d5ca0d33441a4ad074b88dbcb715e/users/f4f3b121e4c444cb961bf9e1c23bf741/roles/36d6d323d9e64e659591dbaea3081bcc` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:33:08.984889 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cae083f-1a0c-4a36-8ccd-d6ecd4bca7bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=310d5ca0d33441a4ad074b88dbcb715e, user_id=f4f3b121e4c444cb961bf9e1c23bf741, role_id=36d6d323d9e64e659591dbaea3081bcc)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:33:08.988650 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cae083f-1a0c-4a36-8ccd-d6ecd4bca7bf admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:33:09.054024 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1437/2891] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:33:08 2022] PUT /identity/v3/projects/310d5ca0d33441a4ad074b88dbcb715e/users/f4f3b121e4c444cb961bf9e1c23bf741/roles/36d6d323d9e64e659591dbaea3081bcc => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:33:09.059108 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2145ca2b-aa4c-481f-a0f7-fd1c758cc8b8 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:33:09.059488 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2145ca2b-aa4c-481f-a0f7-fd1c758cc8b8 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:33:09.059769 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2145ca2b-aa4c-481f-a0f7-fd1c758cc8b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:33:09.078894 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-2145ca2b-aa4c-481f-a0f7-fd1c758cc8b8 None None] MFA Rules not processed for user `329c213a845c4ab2a0642e5a7dbd4b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:33:09.080122 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-2145ca2b-aa4c-481f-a0f7-fd1c758cc8b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:33:09.138472 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1455/2892] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:33:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:33:09.148303 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cae083f-1a0c-4a36-8ccd-d6ecd4bca7bf admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:33:09.150001 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-2cae083f-1a0c-4a36-8ccd-d6ecd4bca7bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:33:09.171792 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:33:09.207588 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cae083f-1a0c-4a36-8ccd-d6ecd4bca7bf admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:33:09.211220 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8a1d99-4c72-4b52-aa7e-09e1b7b2b351 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:33:09.213485 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8a1d99-4c72-4b52-aa7e-09e1b7b2b351 service neutron] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:33:09.213485 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8a1d99-4c72-4b52-aa7e-09e1b7b2b351 service neutron] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:33:09.213485 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8a1d99-4c72-4b52-aa7e-09e1b7b2b351 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:33:09.222211 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8a1d99-4c72-4b52-aa7e-09e1b7b2b351 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:33:09.223741 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8a1d99-4c72-4b52-aa7e-09e1b7b2b351 service neutron] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:33:09.233406 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1438/2893] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:33:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:33:14.293300 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f3902e-5df6-4b73-afad-cba1a6f785fb None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:33:14.293794 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f3902e-5df6-4b73-afad-cba1a6f785fb None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:33:14.293910 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f3902e-5df6-4b73-afad-cba1a6f785fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:33:14.333875 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-e7f3902e-5df6-4b73-afad-cba1a6f785fb None None] MFA Rules not processed for user `f4f3b121e4c444cb961bf9e1c23bf741`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:33:14.335199 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-e7f3902e-5df6-4b73-afad-cba1a6f785fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:33:14.356996 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:33:14.421224 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1456/2894] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:33:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:33:14.426394 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8a1d99-4c72-4b52-aa7e-09e1b7b2b351 service neutron] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:33:14.437868 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8a1d99-4c72-4b52-aa7e-09e1b7b2b351 service neutron] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:33:14.444153 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b34d4f-6757-41db-837e-c706c7c6e8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:33:14.444787 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b34d4f-6757-41db-837e-c706c7c6e8b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:33:14.445073 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b34d4f-6757-41db-837e-c706c7c6e8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:33:14.445347 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b34d4f-6757-41db-837e-c706c7c6e8b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:33:14.445979 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b34d4f-6757-41db-837e-c706c7c6e8b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:33:14.448104 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b34d4f-6757-41db-837e-c706c7c6e8b2 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:33:14.545425 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1439/2895] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 17:33:14 2022] POST /identity/v3/projects => generated 385 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 17:33:14.550261 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f3902e-5df6-4b73-afad-cba1a6f785fb None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:33:14.569733 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f3902e-5df6-4b73-afad-cba1a6f785fb None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:33:14.631420 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5808d27c-7ff9-422f-a0c3-8bf36925848d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:33:14.632042 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5808d27c-7ff9-422f-a0c3-8bf36925848d admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:33:14.632319 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5808d27c-7ff9-422f-a0c3-8bf36925848d admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:33:14.632591 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5808d27c-7ff9-422f-a0c3-8bf36925848d admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:33:14.633216 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5808d27c-7ff9-422f-a0c3-8bf36925848d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:33:14.635216 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5808d27c-7ff9-422f-a0c3-8bf36925848d admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:33:14.742664 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1457/2896] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 17:33:14 2022] POST /identity/v3/users => generated 299 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 17:33:14.746793 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b34d4f-6757-41db-837e-c706c7c6e8b2 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:33:14.756957 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b34d4f-6757-41db-837e-c706c7c6e8b2 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:33:14.761722 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d69cf2e-6b4e-48c1-b0a0-dc2a4e352b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:33:14.762326 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d69cf2e-6b4e-48c1-b0a0-dc2a4e352b13 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:33:14.762611 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d69cf2e-6b4e-48c1-b0a0-dc2a4e352b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:33:14.762892 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d69cf2e-6b4e-48c1-b0a0-dc2a4e352b13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:33:14.763560 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d69cf2e-6b4e-48c1-b0a0-dc2a4e352b13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:33:14.765167 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d69cf2e-6b4e-48c1-b0a0-dc2a4e352b13 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:33:14.776422 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1440/2897] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:33:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 17:33:14.780776 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5808d27c-7ff9-422f-a0c3-8bf36925848d admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:33:14.791755 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5808d27c-7ff9-422f-a0c3-8bf36925848d admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:33:14.796521 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9593cfa4-6242-4375-b07e-debea8fad747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:33:14.797182 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9593cfa4-6242-4375-b07e-debea8fad747 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:33:14.797465 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9593cfa4-6242-4375-b07e-debea8fad747 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:33:14.797742 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9593cfa4-6242-4375-b07e-debea8fad747 admin admin] PATH_INFO: `/v3/projects/a68100ef7e6f4b40bffcbc5285a71002/users/c0feccd7c68d4bc2ae736e0b81af9973/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:33:14.813679 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9593cfa4-6242-4375-b07e-debea8fad747 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a68100ef7e6f4b40bffcbc5285a71002, user_id=c0feccd7c68d4bc2ae736e0b81af9973, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:33:14.816426 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9593cfa4-6242-4375-b07e-debea8fad747 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:33:14.900031 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1458/2898] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:33:14 2022] PUT /identity/v3/projects/a68100ef7e6f4b40bffcbc5285a71002/users/c0feccd7c68d4bc2ae736e0b81af9973/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:33:14.903935 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d69cf2e-6b4e-48c1-b0a0-dc2a4e352b13 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:33:14.914715 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d69cf2e-6b4e-48c1-b0a0-dc2a4e352b13 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:33:14.971931 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7213e94-1f7d-4cc4-94f7-36d20a06af49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:33:14.972559 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7213e94-1f7d-4cc4-94f7-36d20a06af49 admin admin] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:33:14.972839 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7213e94-1f7d-4cc4-94f7-36d20a06af49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:33:14.973110 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7213e94-1f7d-4cc4-94f7-36d20a06af49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:33:14.973741 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7213e94-1f7d-4cc4-94f7-36d20a06af49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:33:14.975369 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7213e94-1f7d-4cc4-94f7-36d20a06af49 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:33:14.984894 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1441/2899] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 17:33:14 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 17:33:14.989065 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9593cfa4-6242-4375-b07e-debea8fad747 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:33:14.999534 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9593cfa4-6242-4375-b07e-debea8fad747 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:33:15.004766 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac0a1c6-a83c-4b0c-8d21-94c74c12445c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:33:15.005426 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac0a1c6-a83c-4b0c-8d21-94c74c12445c admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:33:15.005709 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac0a1c6-a83c-4b0c-8d21-94c74c12445c admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:33:15.005983 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac0a1c6-a83c-4b0c-8d21-94c74c12445c admin admin] PATH_INFO: `/v3/projects/a68100ef7e6f4b40bffcbc5285a71002/users/c0feccd7c68d4bc2ae736e0b81af9973/roles/2c6966cfb10f4d72b0242e9fb5e13b17` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:33:15.007931 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac0a1c6-a83c-4b0c-8d21-94c74c12445c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a68100ef7e6f4b40bffcbc5285a71002, user_id=c0feccd7c68d4bc2ae736e0b81af9973, role_id=2c6966cfb10f4d72b0242e9fb5e13b17)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:33:15.010570 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac0a1c6-a83c-4b0c-8d21-94c74c12445c admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:33:15.021484 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1459/2900] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 17:33:14 2022] PUT /identity/v3/projects/a68100ef7e6f4b40bffcbc5285a71002/users/c0feccd7c68d4bc2ae736e0b81af9973/roles/2c6966cfb10f4d72b0242e9fb5e13b17 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:33:20.475248 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bc6a6b-033c-41e1-aba1-2ff6f46d7d1e None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:33:20.475248 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bc6a6b-033c-41e1-aba1-2ff6f46d7d1e None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:33:20.475248 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bc6a6b-033c-41e1-aba1-2ff6f46d7d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:33:20.510353 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-f7bc6a6b-033c-41e1-aba1-2ff6f46d7d1e None None] MFA Rules not processed for user `c0feccd7c68d4bc2ae736e0b81af9973`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:33:20.511617 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-f7bc6a6b-033c-41e1-aba1-2ff6f46d7d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:33:20.594683 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1442/2901] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:33:20 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:33:20.609205 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac0a1c6-a83c-4b0c-8d21-94c74c12445c admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:33:20.611241 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-2ac0a1c6-a83c-4b0c-8d21-94c74c12445c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:33:20.678461 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac0a1c6-a83c-4b0c-8d21-94c74c12445c admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:33:20.682167 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41a0eae-b63a-4ade-9fb0-06bef00c8cb2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bee1ba6bb324de4915904f83349b3c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:33:20.682816 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41a0eae-b63a-4ade-9fb0-06bef00c8cb2 service glance] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:33:20.683143 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41a0eae-b63a-4ade-9fb0-06bef00c8cb2 service glance] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:33:20.683424 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41a0eae-b63a-4ade-9fb0-06bef00c8cb2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:33:20.716102 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:33:20.751736 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41a0eae-b63a-4ade-9fb0-06bef00c8cb2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:33:20.755979 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41a0eae-b63a-4ade-9fb0-06bef00c8cb2 service glance] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:33:20.766052 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1460/2902] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:33:20 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:33:21.024629 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc8555-787d-451c-9d92-515c0bdbfbf2 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:33:21.024944 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc8555-787d-451c-9d92-515c0bdbfbf2 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:33:21.025216 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc8555-787d-451c-9d92-515c0bdbfbf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:33:21.044201 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:33:21.088538 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-20cc8555-787d-451c-9d92-515c0bdbfbf2 None None] MFA Rules not processed for user `5de19a54ed104bcdb35ee4e5e793f1e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:33:21.089751 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-20cc8555-787d-451c-9d92-515c0bdbfbf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:33:21.156207 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1443/2903] 10.222.0.82 () {58 vars in 1009 bytes} [Thu Aug 18 17:33:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:33:21.314376 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41a0eae-b63a-4ade-9fb0-06bef00c8cb2 service glance] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:33:21.316025 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-d41a0eae-b63a-4ade-9fb0-06bef00c8cb2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:33:21.396970 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41a0eae-b63a-4ade-9fb0-06bef00c8cb2 service glance] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:33:21.402000 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd17138-cccc-41a1-a44b-572b60d28ee5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1381673f81584f4b8084823e3aa99e2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:33:21.402602 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd17138-cccc-41a1-a44b-572b60d28ee5 service swift] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:33:21.402882 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd17138-cccc-41a1-a44b-572b60d28ee5 service swift] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:33:21.403183 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd17138-cccc-41a1-a44b-572b60d28ee5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:33:21.411685 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd17138-cccc-41a1-a44b-572b60d28ee5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:33:21.413248 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd17138-cccc-41a1-a44b-572b60d28ee5 service swift] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:33:21.421512 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1461/2904] 10.222.0.82 () {60 vars in 1384 bytes} [Thu Aug 18 17:33:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 17:33:31.153297 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=39 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:33:31.418638 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=52 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:33:31.452059 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cbd948-9411-4ea6-af1b-9a214f02723c None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:33:31.452348 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cbd948-9411-4ea6-af1b-9a214f02723c None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:33:31.452624 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cbd948-9411-4ea6-af1b-9a214f02723c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:33:31.472525 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-35cbd948-9411-4ea6-af1b-9a214f02723c None None] MFA Rules not processed for user `f4f3b121e4c444cb961bf9e1c23bf741`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:33:31.474251 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-35cbd948-9411-4ea6-af1b-9a214f02723c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:33:31.480441 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1444/2905] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 17:33:31 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:33:31.519720 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd17138-cccc-41a1-a44b-572b60d28ee5 service swift] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:33:31.529425 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd17138-cccc-41a1-a44b-572b60d28ee5 service swift] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:33:31.532989 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f845f0c9-0e04-42d8-9eee-0d0a756ca2ea service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bee1ba6bb324de4915904f83349b3c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:33:31.533570 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f845f0c9-0e04-42d8-9eee-0d0a756ca2ea service glance] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:33:31.534502 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f845f0c9-0e04-42d8-9eee-0d0a756ca2ea service glance] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:33:31.534772 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f845f0c9-0e04-42d8-9eee-0d0a756ca2ea service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:33:31.536500 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-f845f0c9-0e04-42d8-9eee-0d0a756ca2ea service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:33:31.608002 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f845f0c9-0e04-42d8-9eee-0d0a756ca2ea service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:33:31.609419 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f845f0c9-0e04-42d8-9eee-0d0a756ca2ea service glance] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:33:31.620752 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1462/2906] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:33:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:33:31.844931 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cbd948-9411-4ea6-af1b-9a214f02723c None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:33:31.846928 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-35cbd948-9411-4ea6-af1b-9a214f02723c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:33:31.902176 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cbd948-9411-4ea6-af1b-9a214f02723c None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:33:31.905706 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6ca7f0-8fd2-40bc-b93a-c6c0f5b55007 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:33:31.906306 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ca7f0-8fd2-40bc-b93a-c6c0f5b55007 service neutron] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:33:31.906589 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ca7f0-8fd2-40bc-b93a-c6c0f5b55007 service neutron] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:33:31.906860 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ca7f0-8fd2-40bc-b93a-c6c0f5b55007 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:33:31.908562 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-3e6ca7f0-8fd2-40bc-b93a-c6c0f5b55007 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:33:31.918615 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6ca7f0-8fd2-40bc-b93a-c6c0f5b55007 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:33:31.920085 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6ca7f0-8fd2-40bc-b93a-c6c0f5b55007 service neutron] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:33:31.928618 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1445/2907] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:33:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:33:32.317764 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f845f0c9-0e04-42d8-9eee-0d0a756ca2ea service glance] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:33:32.327814 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f845f0c9-0e04-42d8-9eee-0d0a756ca2ea service glance] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:33:32.332031 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09825cea-d986-4d2b-bd25-6bfb480de850 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:33:32.332635 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09825cea-d986-4d2b-bd25-6bfb480de850 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:33:32.332917 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09825cea-d986-4d2b-bd25-6bfb480de850 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:33:32.333191 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09825cea-d986-4d2b-bd25-6bfb480de850 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:33:32.334948 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-09825cea-d986-4d2b-bd25-6bfb480de850 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:33:32.349258 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09825cea-d986-4d2b-bd25-6bfb480de850 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:33:32.350847 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09825cea-d986-4d2b-bd25-6bfb480de850 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:33:32.364760 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1463/2908] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:33:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:33:34.986308 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74367a-da1a-45b7-9027-768a574dc360 None None] REQUEST_METHOD: `POST` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:33:34.986816 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74367a-da1a-45b7-9027-768a574dc360 None None] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:33:34.987145 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74367a-da1a-45b7-9027-768a574dc360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:33:35.006430 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.auth.core [None req-ae74367a-da1a-45b7-9027-768a574dc360 None None] MFA Rules not processed for user `5de19a54ed104bcdb35ee4e5e793f1e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81650) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:33:35.007808 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-ae74367a-da1a-45b7-9027-768a574dc360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:33:35.015031 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1446/2909] 10.222.0.82 () {58 vars in 1009 bytes} [Thu Aug 18 17:33:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:33:35.028735 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09825cea-d986-4d2b-bd25-6bfb480de850 service neutron] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:33:35.041036 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09825cea-d986-4d2b-bd25-6bfb480de850 service neutron] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:33:35.046078 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b881359c-2102-4b0f-9f5f-2da4aa65b6b5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1381673f81584f4b8084823e3aa99e2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:33:35.046758 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b881359c-2102-4b0f-9f5f-2da4aa65b6b5 service swift] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:33:35.047145 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b881359c-2102-4b0f-9f5f-2da4aa65b6b5 service swift] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:33:35.047498 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b881359c-2102-4b0f-9f5f-2da4aa65b6b5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:33:35.056501 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b881359c-2102-4b0f-9f5f-2da4aa65b6b5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:33:35.058067 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b881359c-2102-4b0f-9f5f-2da4aa65b6b5 service swift] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:33:35.066811 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1464/2910] 10.222.0.82 () {60 vars in 1384 bytes} [Thu Aug 18 17:33:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 17:33:45.000742 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=32 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:33:45.062838 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=46 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:33:45.508574 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae74367a-da1a-45b7-9027-768a574dc360 None None] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:33:45.520049 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae74367a-da1a-45b7-9027-768a574dc360 None None] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:33:45.524599 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f6eb74-b1da-4f7c-8efb-188efbcc1853 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:33:45.525008 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f6eb74-b1da-4f7c-8efb-188efbcc1853 service neutron] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:33:45.525289 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f6eb74-b1da-4f7c-8efb-188efbcc1853 service neutron] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:33:45.526299 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f6eb74-b1da-4f7c-8efb-188efbcc1853 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:33:45.528439 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-48f6eb74-b1da-4f7c-8efb-188efbcc1853 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:33:45.540855 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f6eb74-b1da-4f7c-8efb-188efbcc1853 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:33:45.543118 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f6eb74-b1da-4f7c-8efb-188efbcc1853 service neutron] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:33:45.552831 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1447/2911] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:33:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:33:55.548062 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=6 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:34:04.778342 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b881359c-2102-4b0f-9f5f-2da4aa65b6b5 service swift] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:34:04.788484 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b881359c-2102-4b0f-9f5f-2da4aa65b6b5 service swift] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:34:04.792051 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed114a7-8211-4123-b7ea-73bae4623892 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2403ede53148b2a113ffa2a605e987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:34:04.793104 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed114a7-8211-4123-b7ea-73bae4623892 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:34:04.793104 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed114a7-8211-4123-b7ea-73bae4623892 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:34:04.793338 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed114a7-8211-4123-b7ea-73bae4623892 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:34:04.802173 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed114a7-8211-4123-b7ea-73bae4623892 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:34:04.803616 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed114a7-8211-4123-b7ea-73bae4623892 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:34:04.812770 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1465/2912] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:34:04 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:34:14.808662 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=6 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:35:46.446950 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f6eb74-b1da-4f7c-8efb-188efbcc1853 service neutron] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:35:46.448395 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-48f6eb74-b1da-4f7c-8efb-188efbcc1853 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:35:46.516125 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f6eb74-b1da-4f7c-8efb-188efbcc1853 service neutron] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:35:46.521819 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e4c9b2-55d3-4adc-be18-762d08c851d2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29653ba1a2a74232972ef7a9000518b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:35:46.522391 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e4c9b2-55d3-4adc-be18-762d08c851d2 service placement] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:35:46.522669 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e4c9b2-55d3-4adc-be18-762d08c851d2 service placement] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:35:46.522971 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e4c9b2-55d3-4adc-be18-762d08c851d2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:35:46.524681 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-53e4c9b2-55d3-4adc-be18-762d08c851d2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:35:46.536014 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e4c9b2-55d3-4adc-be18-762d08c851d2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:35:46.537838 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e4c9b2-55d3-4adc-be18-762d08c851d2 service placement] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:35:46.547639 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1448/2913] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:35:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:35:56.543441 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=32 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:36:32.995379 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ac138-202b-44c2-ba05-fb30670973ea None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:36:32.995881 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ac138-202b-44c2-ba05-fb30670973ea None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:36:32.995985 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ac138-202b-44c2-ba05-fb30670973ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:36:33.015715 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-478ac138-202b-44c2-ba05-fb30670973ea None None] MFA Rules not processed for user `5de19a54ed104bcdb35ee4e5e793f1e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 17:36:33.016944 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-478ac138-202b-44c2-ba05-fb30670973ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:36:33.024471 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1466/2914] 10.222.0.82 () {58 vars in 967 bytes} [Thu Aug 18 17:36:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 17:36:33.064917 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e4c9b2-55d3-4adc-be18-762d08c851d2 service placement] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:36:33.077103 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e4c9b2-55d3-4adc-be18-762d08c851d2 service placement] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:36:33.082066 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c222d07d-8fbe-4d66-86bc-756cca1ea8cc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1381673f81584f4b8084823e3aa99e2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:36:33.082682 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c222d07d-8fbe-4d66-86bc-756cca1ea8cc service swift] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:36:33.082998 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c222d07d-8fbe-4d66-86bc-756cca1ea8cc service swift] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:36:33.083315 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c222d07d-8fbe-4d66-86bc-756cca1ea8cc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:36:33.091188 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c222d07d-8fbe-4d66-86bc-756cca1ea8cc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:36:33.092765 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c222d07d-8fbe-4d66-86bc-756cca1ea8cc service swift] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:36:33.100755 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1449/2915] 10.222.0.82 () {60 vars in 1384 bytes} [Thu Aug 18 17:36:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 17:36:40.838310 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478ac138-202b-44c2-ba05-fb30670973ea None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:36:40.852940 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478ac138-202b-44c2-ba05-fb30670973ea None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:36:40.859409 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c219922-ef1f-4cc3-909f-3d39e11261c7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29653ba1a2a74232972ef7a9000518b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:36:40.865011 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c219922-ef1f-4cc3-909f-3d39e11261c7 service placement] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:36:40.865011 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c219922-ef1f-4cc3-909f-3d39e11261c7 service placement] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:36:40.865011 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c219922-ef1f-4cc3-909f-3d39e11261c7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:36:40.865011 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-3c219922-ef1f-4cc3-909f-3d39e11261c7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:36:40.882008 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c219922-ef1f-4cc3-909f-3d39e11261c7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:36:40.883610 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c219922-ef1f-4cc3-909f-3d39e11261c7 service placement] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:36:40.895377 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1467/2916] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:36:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:36:41.002317 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c222d07d-8fbe-4d66-86bc-756cca1ea8cc service swift] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:36:41.068353 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c222d07d-8fbe-4d66-86bc-756cca1ea8cc service swift] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:36:41.073370 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bf3719-7b28-46de-a976-31cec81e1ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:36:41.082945 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf3719-7b28-46de-a976-31cec81e1ede admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:36:41.082945 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf3719-7b28-46de-a976-31cec81e1ede admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:36:41.082945 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf3719-7b28-46de-a976-31cec81e1ede admin admin] PATH_INFO: `/v3/users/f4f3b121e4c444cb961bf9e1c23bf741` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:36:41.082945 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bf3719-7b28-46de-a976-31cec81e1ede admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4f3b121e4c444cb961bf9e1c23bf741)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:36:41.082945 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bf3719-7b28-46de-a976-31cec81e1ede admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:36:41.237173 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-f9bf3719-7b28-46de-a976-31cec81e1ede admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4f3b121e4c444cb961bf9e1c23bf741'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:36:41.249106 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-f9bf3719-7b28-46de-a976-31cec81e1ede admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4f3b121e4c444cb961bf9e1c23bf741'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:36:41.251856 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-f9bf3719-7b28-46de-a976-31cec81e1ede admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4f3b121e4c444cb961bf9e1c23bf741'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:36:41.331808 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-f9bf3719-7b28-46de-a976-31cec81e1ede admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4f3b121e4c444cb961bf9e1c23bf741'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:36:41.332465 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-f9bf3719-7b28-46de-a976-31cec81e1ede admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4f3b121e4c444cb961bf9e1c23bf741'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:36:41.345416 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1450/2917] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:36:41 2022] DELETE /identity/v3/users/f4f3b121e4c444cb961bf9e1c23bf741 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:36:41.574378 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c219922-ef1f-4cc3-909f-3d39e11261c7 service placement] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:36:41.585456 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c219922-ef1f-4cc3-909f-3d39e11261c7 service placement] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:36:41.644439 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752c90c5-8b39-4f56-a5a1-fb72ce97a99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:36:41.645071 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752c90c5-8b39-4f56-a5a1-fb72ce97a99c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:36:41.645354 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752c90c5-8b39-4f56-a5a1-fb72ce97a99c admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:36:41.645631 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752c90c5-8b39-4f56-a5a1-fb72ce97a99c admin admin] PATH_INFO: `/v3/projects/310d5ca0d33441a4ad074b88dbcb715e` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:36:41.646771 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752c90c5-8b39-4f56-a5a1-fb72ce97a99c admin admin] RBAC: Authorizing `identity:delete_project(project_id=310d5ca0d33441a4ad074b88dbcb715e)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:36:41.649272 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752c90c5-8b39-4f56-a5a1-fb72ce97a99c admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:36:41.754593 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-752c90c5-8b39-4f56-a5a1-fb72ce97a99c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '310d5ca0d33441a4ad074b88dbcb715e', 'request_id': 'req-752c90c5-8b39-4f56-a5a1-fb72ce97a99c'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:36:41.761381 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-752c90c5-8b39-4f56-a5a1-fb72ce97a99c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '310d5ca0d33441a4ad074b88dbcb715e', 'request_id': 'req-752c90c5-8b39-4f56-a5a1-fb72ce97a99c'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:36:41.809408 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-752c90c5-8b39-4f56-a5a1-fb72ce97a99c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '310d5ca0d33441a4ad074b88dbcb715e', 'request_id': 'req-752c90c5-8b39-4f56-a5a1-fb72ce97a99c'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:36:41.820755 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-752c90c5-8b39-4f56-a5a1-fb72ce97a99c admin admin] The token cache is being invalidate because project 310d5ca0d33441a4ad074b88dbcb715e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:36:41.821040 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-752c90c5-8b39-4f56-a5a1-fb72ce97a99c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:36:41.821659 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-752c90c5-8b39-4f56-a5a1-fb72ce97a99c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:36:41.823199 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1468/2918] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:36:41 2022] DELETE /identity/v3/projects/310d5ca0d33441a4ad074b88dbcb715e => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:36:41.827181 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bf3719-7b28-46de-a976-31cec81e1ede admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:36:41.828578 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-f9bf3719-7b28-46de-a976-31cec81e1ede admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:36:41.841723 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bf3719-7b28-46de-a976-31cec81e1ede admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:36:41.898353 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea2dad1-c25f-4561-99cc-124b689375b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:36:41.899009 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea2dad1-c25f-4561-99cc-124b689375b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:36:41.899292 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea2dad1-c25f-4561-99cc-124b689375b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:36:41.899567 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea2dad1-c25f-4561-99cc-124b689375b2 admin admin] PATH_INFO: `/v3/users/c0feccd7c68d4bc2ae736e0b81af9973` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:36:41.900688 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea2dad1-c25f-4561-99cc-124b689375b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0feccd7c68d4bc2ae736e0b81af9973)` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:36:41.902613 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea2dad1-c25f-4561-99cc-124b689375b2 admin admin] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:36:41.907810 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:36:42.075289 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-8ea2dad1-c25f-4561-99cc-124b689375b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0feccd7c68d4bc2ae736e0b81af9973'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:36:42.087288 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-8ea2dad1-c25f-4561-99cc-124b689375b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0feccd7c68d4bc2ae736e0b81af9973'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:36:42.087970 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-8ea2dad1-c25f-4561-99cc-124b689375b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0feccd7c68d4bc2ae736e0b81af9973'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:36:42.143095 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-8ea2dad1-c25f-4561-99cc-124b689375b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0feccd7c68d4bc2ae736e0b81af9973'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:36:42.143653 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.notifications [None req-8ea2dad1-c25f-4561-99cc-124b689375b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0feccd7c68d4bc2ae736e0b81af9973'} {{(pid=81650) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:36:42.156179 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1451/2919] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 17:36:41 2022] DELETE /identity/v3/users/c0feccd7c68d4bc2ae736e0b81af9973 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:36:42.394135 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752c90c5-8b39-4f56-a5a1-fb72ce97a99c admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:36:42.395603 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-752c90c5-8b39-4f56-a5a1-fb72ce97a99c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:36:42.411175 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752c90c5-8b39-4f56-a5a1-fb72ce97a99c admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:36:42.469605 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80218013-0f7e-4a2b-8e86-26a5cd536b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c213a845c4ab2a0642e5a7dbd4b18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77b6b270950a499cb6b0da7c630c03b6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:36:42.476280 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80218013-0f7e-4a2b-8e86-26a5cd536b2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:36:42.476280 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80218013-0f7e-4a2b-8e86-26a5cd536b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:36:42.476280 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80218013-0f7e-4a2b-8e86-26a5cd536b2e admin admin] PATH_INFO: `/v3/projects/a68100ef7e6f4b40bffcbc5285a71002` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:36:42.476280 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80218013-0f7e-4a2b-8e86-26a5cd536b2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a68100ef7e6f4b40bffcbc5285a71002)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:36:42.476280 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80218013-0f7e-4a2b-8e86-26a5cd536b2e admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:36:42.485158 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:36:42.576252 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-80218013-0f7e-4a2b-8e86-26a5cd536b2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a68100ef7e6f4b40bffcbc5285a71002', 'request_id': 'req-80218013-0f7e-4a2b-8e86-26a5cd536b2e'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:36:42.582343 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-80218013-0f7e-4a2b-8e86-26a5cd536b2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a68100ef7e6f4b40bffcbc5285a71002', 'request_id': 'req-80218013-0f7e-4a2b-8e86-26a5cd536b2e'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:36:42.660922 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-80218013-0f7e-4a2b-8e86-26a5cd536b2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a68100ef7e6f4b40bffcbc5285a71002', 'request_id': 'req-80218013-0f7e-4a2b-8e86-26a5cd536b2e'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:36:42.672031 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-80218013-0f7e-4a2b-8e86-26a5cd536b2e admin admin] The token cache is being invalidate because project a68100ef7e6f4b40bffcbc5285a71002 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 17:36:42.675352 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-80218013-0f7e-4a2b-8e86-26a5cd536b2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:36:42.675352 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-80218013-0f7e-4a2b-8e86-26a5cd536b2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 17:36:42.675352 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1469/2920] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 17:36:42 2022] DELETE /identity/v3/projects/a68100ef7e6f4b40bffcbc5285a71002 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 17:36:52.152954 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:36:52.668775 dv-cinder-853679-1-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=19,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 17:40:49.595228 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea2dad1-c25f-4561-99cc-124b689375b2 admin admin] Authenticating user token {{(pid=81650) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 17:40:49.596672 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-8ea2dad1-c25f-4561-99cc-124b689375b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:40:49.709473 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea2dad1-c25f-4561-99cc-124b689375b2 admin admin] Validating token access rules against request {{(pid=81650) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 17:40:49.768670 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105ecaf0-5782-40c0-8bb1-30985e3515f0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29653ba1a2a74232972ef7a9000518b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2293c0aab61f44849d0d48b214cde7ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81650) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 17:40:49.769492 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ecaf0-5782-40c0-8bb1-30985e3515f0 service placement] REQUEST_METHOD: `GET` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 17:40:49.769604 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ecaf0-5782-40c0-8bb1-30985e3515f0 service placement] SCRIPT_NAME: `/identity` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 17:40:49.769832 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ecaf0-5782-40c0-8bb1-30985e3515f0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81650) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 17:40:49.771713 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.fernet_utils [None req-105ecaf0-5782-40c0-8bb1-30985e3515f0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81650) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 17:40:49.783419 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105ecaf0-5782-40c0-8bb1-30985e3515f0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 17:40:49.785025 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105ecaf0-5782-40c0-8bb1-30985e3515f0 service placement] RBAC: Authorization granted {{(pid=81650) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 17:40:49.794977 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: [pid: 81650|app: 0|req: 1452/2921] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 17:40:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 17:40:59.790697 dv-cinder-853679-1-iscsi devstack@keystone.service[81650]: DEBUG dbcounter [-] [81650] Writing DB stats keystone:SELECT=77 {{(pid=81650) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}